Archive

Archive for the ‘tools’ Category

14 useful ARP monitoring tools

February 13th, 2010 madiga No comments

Some say ARP is an old school crap that it’s no longer useful in this modern day and age.  Those who say this, don’t know what it’s all about.  ARP is used to link IP address to a system’s physical MAC address in a local network, this is how the servers identify each other.

By understanding ARP and knowing how to use the arp utility, one can troubleshoot network related issues faster.  In this article, we’ve put together 14 tools specifically used to to deal with ARP related monitoring and troubleshooting.

1) Arping

- an ARP level ping utility. It’s good for finding out if an IP is taken before you have routing to that subnet. It can also ping MAC addresses directly.

2) arp-scan

-  sends ARP (Address Resolution Protocol) queries to the specified targets, and displays any responses that are received. It allows any part of the outgoing ARP packets to be changed, allowing the behavior of targets to non-standard ARP packets to be examined. The IP address and hardware address of received packets are displayed, together with the vendor details. These details are obtained from the IEEE OUI and IAB listings, plus a few manual entries. It includes arp-fingerprint, which allows a system to be fingerprinted based on how it responds to non-standard ARP packets.

3)  arpalert

- uses ARP address monitoring to help prevent unauthorized connections on the local network. If an illegal connection is detected, a program or script is launched, which could be used to send an alert message, for example.

4)  parprouted

- a daemon for transparent IP (Layer 3) proxy ARP bridging. This is useful for creation of transparent firewalls and bridging networks with different MAC protocols. Also, unlike standard bridging, proxy ARP bridging allows to bridge Ethernet networks behind wireless nodes without using WDS or layer 2 bridging.

5)  ARPSpoofDetector

-performs active and passive detection of ARP spoofing and IP (IPv4) address collision. The program can send healing packets with regular ARP information.

6)  Local IP Takeover

- provides network link redundancy within a single server that has multiple network interface cards (NICs) with each NIC connected to separate network switches. If the primary NIC fails (i.e. it cannot ping its default gateway), the “service” IP (the IP that the outside world connects to) will automatically float to the secondary NIC and a specially crafted ARP (utilizing send_arp) will be broadcast on the local network, thereby instructing all other hosts to update their local ARP cache. The result is minimal service downtime. Plus, no manual intervention is required in the event that a network card, cable, or switch breaks.

7) ARP Tools

- Collection of libnet and libpcap based ARP utilities. It currently contains ARP Discover (arpdiscover), an Ethernet scanner based on ARP protocol; ARP Flood (arpflood), an ARP request flooder; and ARP Poison (arppoison), for poisoning switches’ MAC address tables.

8 )  Gnome ARP

- an ARP monitoring program written on Gnome with the GTK toolkit and Ruby. It takes ARP tables and some system variables via SNMP and ARP protocols and determines whether any machines have changed their IP address. It is useful for detecting new machines on the network and detecting which machine have changed addresses. It is intended especially for network admins.

9) Arphound

- a tool that listens to all traffic on an ethernet network interface. It reports IP/MAC address pairs as well as events such as IP conflicts, IP changes, IP addresses with no RDNS, various ARP spoofing, and packets not using the expected gateway. Reporting is done to stdout, to a specified file, or to syslog in a format that can be easily parsed by scripts.

10)  wakearp

- a small utility to induce ARP resolution for any listening IP address in the local /24 subnet.

11)  MasarLabs NoArp

- a Linux kernel module that filters and drops unwanted ARP requests. It is useful when you need to add an alias to the loopback interface to use a load balancer.

12)  Antidote

- a detector for ARP poisoning on a switched network.

13)  arprelease

- a small libnet-based tool to flush ARP cache entries from devices like Cisco routers to move an IP from one Linux box to another.

14)  ARPoison

- a network analysis tool that sends ARP packets to/from specified hardware and protocol addresses.

Categories: tools Tags:

15 Remote Desktop Solutions for Linux.

February 7th, 2010 madiga 10 comments

There are a wide range of remote desktop applications that are available that can be used to connect to Windows environment but there aren’t too many that can be used to remote desktop from Linux to Linux or Windows to Linux. With this I mean, getting entire desktop of remote Linux environment on your local workstation.

Most people who are used to a Unix-style environment know that a machine can be reached over the network at the shell level using utilities like telnet or ssh. And some people realize that X Windows output can be redirected back to the client workstation. But many people don’t realize that it is easy to use an entire desktop over the network. There are a several of open source applications that can be used to achieve this.

1)  VNC (Virtual Network Computing) is a remote display system which allows the user to view the desktop of a remote machine anywhere on the internet. It can also be directed through SSH for security.

Basically you install VNC server on the server and install client on your local PC. Setup is extremely easy and server is very stable. On client side, you can set the resolution and connect to IP of VNC server. It can be a bit slow compared to Windows remote desktop and also has the tendency to take more time refreshing over low-bandwidth links. All in all VNC is an amazing piece of free software that gets the job done.

There is RealVNC , TightVNC and UltraVNC. Each has it’s advantages and disadvantages. Most popular one is RealVNC but if you’re upto it, experiment with all three and choose the one that works for you best. By default, communication between client and server is in clear text on port 5900. However, you can easily route all traffic via SSH tunnel. Here is a quick way of setting it up if you have access to command line shell:

ssh -ND 5900 <user>@remote.server.com

When you get prompted, enter your password. Pop open VNC client and connect to ‘localhost’. This’ll route your connection to VNC server on remote machine.

You can download VNC from:

2)  Then there is FreeNX. FreeNX is a system that allows you to access your desktop from another machine over the internet. You can use this to login graphically to your desktop from a remote location. One example of its use would be to have a FreeNX server set up on your home computer, and graphically logging in to the home computer from your work computer, using a FreeNX client. It provides near local speed application responsiveness over high latency, low bandwidth links.

FreeNX can be configured to run via SSH without any tunneling. It binds to your existing SSH install. Instead of guiding you through the installation of FreeNX in this article, you can visit the following URLs that’ll guide you through the installation on Ubuntu:

3) The third free application is 2X Terminal Server for Linux. 2X TerminalServer for Linux is an Open Source project, licensed under the GPL and is free of charge. As far as performance goes, NoMachine’s technology is on par with Windows’ own Remote Desktop Protocol (RDP) suite, better than VNC. Both X2 and FreeNX is based on NoMachine technology.

Here are some quick links if you’re interested in using this software:

4) Then there is is XDMCP. The X Display Manager Control Protocol uses UDP port 177. Compared to the list above, it’s not as easy to setup for remote desktop but it’s the original way of doing this on Linux. You can get setup instructions and other tips in the following URL:

5) CygwinX. A complete Linux emulation on Windows. You’ll find every tool and app that you have on Linux on Cygwin.

6) XRDP. RDP server that runs on Linux, thus allowing you to use Windows Remote Desktop Client or rdesktop to connect.

7) x2vnc – great little utility that allows you to tie a linux and windows (or anything that can run the vncserver) together with a ingle keyboard/mouse, avoiding the need for a switcher box. Mousing cross screens transparently switches between machines, and cut and aste works.

8 ) Xming – t’s a great and lightweight implementation of X11 for Windows that allows you to connect to a Linux box.

9) KDE Desktop Sharing (formerly krfb) – part of KDE since version 3.1. It is located in the kdenetwork package. If your distribution splits the KDE applications into separate packets, you may find the client as ‘krdc’ and the server as ‘krfb’. Also uses VNC technology.

10) X-Win32 - Top rated PC X server solutions for Windows PCs connecting to remote Unix and Linux host systems. Works well over SSH.

11) Single Click UltraVNC – In case you would like to remote control without any software installed on the target computer you need UltraVNC SC. The user on the to be controlled computer needs to simply click on a web page and remote controlling begins.

12) CrossLoop – CrossLoop is a FREE secure screen sharing utility designed for people of all technical skill levels. CrossLoop extends the boundaries of VNC’s traditional screen sharing by enabling non-technical users to get connected from anywhere on the Internet in seconds without changing any firewall or router settings.

13) Thinstation – Although not a remote desktop app but worth mentioning here. Thin client linux distro for terminals using std. x86 hw. It can boot from network, pxe, syslinux,loadlin, CD, floppy or flash-disk and connect to servers using VNC, RDP, XDM, SSH and etc.

14) rdesktop - an open source client for Windows NT Terminal Server and Windows 2000/2003 Terminal Services, capable of natively speaking Remote Desktop Protocol (RDP) in order to present the user’s NT desktop. rdesktop currently runs on most UNIX based platforms with the X Window System, and other ports should be fairly straightforward.

While you’re at it, get grdesktop from (http://www.nongnu.org/grdesktop/). It is a GNOME frontend, for rdesktop. It can save several connections (including their options), and browse the network for available terminal servers.

15) ssh -X – You can check out this great article written by a slashdot user sometime ago.

Windows to Mac / Mac to Windows

1) RDP Client for Mac allows you to connect to a Windows-based computer and work with programs and files on that computer from your Macintosh computer.

2) OSXVnc – Vine Server is a full featured VNC server for Mac OS X providing remote access to the GUI, keyboard and mouse using Vine Viewer or any other VNC client.

3) Chicken of the VNC – A VNC client allows one to display and interact with a remote computer screen. In other words, you can use Chicken of the VNC to interact with a remote computer as though it’s right next to you.

Unfortunately I was not able to find too many available to connect to Mac from Windows other than VNC. I think Windows need to support RDP into Mac. Many people would benefit from this.

If I am missing anything from the list, please let me know.

Categories: linux, tools Tags:

Cool Desktop Multiplier

February 4th, 2010 madiga No comments

I came across this neat application called Userful Desktop Multiplier. It is a virtualized X server that turns one computer into ten “workstations” by using extra video cards, keyboards, and monitors. This approach offers significantly higher performance and lower hardware costs than Thin Client or LTSP.

It installs on most popular Linux distributions (Red Hat, Novell/SuSE, Fedora, Mandriva, Xandros, Linspire, Ubuntu, etc.). It supports USB touch screens (Elotouch, Microtouch), card-swipes, barcode scanners, as well as all virtually all video cards supported by X. It features easy setup and graphical configuration. Download it from here.

Desktop Multiplier

Categories: linux, tools Tags:

Torrent tools for Linux

January 30th, 2010 madiga No comments

In this article, we’ve put together a short list of torrent tools available to make your torrenting experience more interesting.

  1. Torrent Episode Downloader – a smart tool for using BitTorrent to automatically download the newest episodes of TV shows that you choose. It has built-in support for “24″, “Lost”, “The Simpsons”, “South Park”, and many more. You can also define new shows to download. It checks feeds from TorrentReactor, Mininova, TVRSS, The PirateBay, Mr. Twig, Isohunt, and MyBittorrent for new episodes of TV shows.
  2. Torrent Pouncer – an automatic torrent downloader for periodic releases of TV shows, series, and similar. Contrary to popular RSS-based torrent downloaders, it searches various torrent search engines like Mininova and Pirate Bay. For user convenience, it keeps databases of downloaded torrents and associates them with shows and season and episode numbers.
  3. TorrentSniff – Reports current status information on a BitTorrent torrent including what files it contains and the current number of seeds (machines sharing the complete torrent) and leeches (machines still downloading the torrent). It is useful for comparing multiple potential sources of a file to find the best-seeded torrent.
  4. Torrent Swapper – A sociable peer to peer file-sharing client based on the BitTorrent protocol. It has a basic understanding of human friendships, of user tastes in content, and of Internet connectivity between users. Torrent Swapper boasts unique features like Amazon-like recommendations to get interesting files, doubling the download speed by using the upload capacity of friends, real-time P2P file sharing with P2P video streaming, and showing the locations of seeders and leechers of the same content with city-level accuracy on a world map.
  5. Bitflu – BitTorrent client designed to run non-stop as a daemon. It does not provide a graphical interface, but offers a telnet interface and can handle multiple torrent downloads
Categories: tools Tags:

Grep your network traffic!

January 21st, 2010 madiga No comments

There are so many different tools available for network troubleshooting such as tcpdump. However, in this article, I’ll discuss something that’s easy to learn but powerful enough to troubleshoot real-time traffic in and out of your Linux server.

ngrep uses the libpcap library, and can also take hexadecimal expressions for which to capture network traffic. Before installing, make sure you have the libpcap library and of course ngrep. You can run ngrep only as root and running without any option, it will listen to all traffic on the current interface.

Below are some examples:

# ngrep '' udp

- Print packets matching a particular protocol, in this case only UDP. You can exchange udp to tcp to grap tcp data.

# ngrep '' port 53

- Shows all DNS requests

# ngrep 'SSH' port 22

- Displays all connections to port 22

# ngrep 'user' port 110

- Grabs pop info on ‘user’.

# ngrep digg.com port 80

- With this command running in bg, you can tell which user is access digg.com as well as if digg.com is connecting to your server.  This becomes handy if for example for large hosting site.  If one of the sites on your server is getting is under digg effect, instead of going through every site’s access_log, this short-cut will lead you to the culprit in no time.

# ngrep -qd eth1 'www' tcp port 80

- Look only at the tcp packets of port 80 via interface eth1 for anything matching ‘www’.

# ngrep -iq 'rcpt to|mail from' tcp port 25

- With the above command, you can monitor current email transactions and print the addresses.

# ngrep -q -t port 21

- The above command captures all traffic to the FTP server (port 21).

# ngrep -t '^(GET|POST) ' 'src host 192.168.136.55 and tcp and dst port 80'

- Displays all outgoing web requests from 192.168.136.55.

# ngrep -t 'USER' 'tcp and port 110'

- Displays in clear text, who is logging in to their pop accounts.

# ngrep -iq 'user-agent' tcp port 80

- Displays the browser type the client host is running.

And finally doing a man ngrep and ngrep –help provide much more options and explanations on best ways to use ngrep.  Ngrep is a great tool but it has it’s limitations which is why tools like tcpdump and others come into the picture to help out.

Categories: tools Tags:

10 tools to recover deleted files in Linux

January 19th, 2010 madiga No comments

If you’re like me, you mus have at some point deleted a file or directory that you immediately regretted after.  On Windows, it’s easy to recover…just go to recycle bin and restore but on Linux, how do you do it?!

Some distros of Linux has recycle bin such as Ubuntu which you can restore but for the majority and deleting via command-line, the deleted files seem to disappear forever and without any trace that it ever existed.

In this article, I’ve put together list of techniques and tools that can do file/directory recovery on Linux platform. But before I start listing the tools, a few common sense techniques which can make it a bit harder to delete files on your system.

For example, change rm to alias=’rm -i’. This way, every time you delete a file, the system will ask if you’re sure.  Second thing is simply to do backups.  If you don’t have tape, use a utility like rsync to copy the important directories and files to another system or partition. You can set a crontab to do this daily or even hourly. Makes restoring so much easier in case you delete something by mistake.

  1. Recover – automates some steps as described in the Ext2fs-Undeletion-howto in order to recover a lost file. This utility will greatly increase your file recover rate. People who do not know how to undelete a file are encouraged to use this utility.

  2. athena-delete – was written at Project Athena in response to the problem which many novice UNIX users have with accidentally deleting files they didn’t want to delete.

  3. unrm – a small shell utility which can, under some circumstances, recover almost 99% of your erased data (similar to DOS’s undelete). Read carefully the FAQ file and preferably the Linux Ext2fs Undeletion Mini-HOWTO before using it.

  4. gET iT i sAY – a file recovery tool for Ext2/Ext3 filesystems. Once installed, current files and newly created files from /root and /home are recovered. It allows users to recover all deleted files, recover files owned by a specific user, dump data from old file locations, and recover files of a specific type, such as text or MP3. A forensic analyzer is also provided to assist users during recovery.

  5. e2undel – an interactive console tool that recovers the data of deleted files on an ext2 file system under Linux. A library that allows you to recover deleted files by name is included. e2undel does not manipulate any internal ext2 structures, and it does not require any additional tools. It should be useable without knowledge of the ext2 internal.

  6. anyfs-tools – allows you to recover and convert filesystems while using minimal extra disk space. Unlike other recovery tools, anyfs-tools doesn’t copy all the files found on other disks (or partitions), but only saves information about the location of the file blocks to an external inode table. After recovery, a user may mount the damaged filesystem with anyfs and the external inode table and then interact with all recovered files with any program.

  7. rfs – a shell script for creating and updating a local spare system disk. The main goal is to recover a working system after a crash quickly. In this case, “quickly” means the time it takes to reboot the machine. rfs stands for “replication of filesystems”. Like rsyncbackup, rfs is built on top of rsync.

  8. e2retrieve - an Ext2 data recovery tool that works on truncated or partial filesystems. It is very useful to retrieve data when a disk from an LVM crashes. It does not repair the filesystem but extracts and copies most of the data it can from raw Ext2 data given to it.

  9. findfile – a collection of tools for recovering files from filesystems with corrupted directories, allocation tables, etc. This can be useful when the partition table (or more) of a hard disk is corrupted, or when you have a memory card of a digital camera which is corrupted.

  10. TestDisk – a tool to check and undelete partitions. It works with the following partitions: FAT12, FAT16, FAT32, Linux, Linux swap (version 1 and 2), NTFS (Windows NT/W2K/2003), BeFS (BeOS), UFS (BSD), JFS, XFS, and Netware.

Some of these tools are old but can still be useful. If we missed something, please let us know about it.

Categories: tools Tags:

fsck Alternatives

January 16th, 2010 madiga 1 comment

We’ve all been there. Our Linux system crashing unexpectedly either by loss of power, you tripping on power cord, your cat or 2 year old son tripping on power cord etc. The end result is your PC shutting off and caught completely by surprise.

Perhaps a better analogy is you’re Linux box quietly humming along, reading some news, browsing around while taking a nice long crap…all of a sudden, someone, in this case you, your cat or some other external force push it off the toilet while it was about to drop the biggest crap of all.

You surprised the heck out of it and made it crap on itself instead of the toilet and now it’s a big mess. This the case when your server unexpectedly crashes and you have to clean up the mess before it can boot up cleanly again.

fsck command is the most common way of dealing with such issues, however it can really test your patience by taking the longest time to recover a disk/partition, not recover at first try or at all, continuous bad syntax and so much frustration. Thankfully Linux has 4 other major filesystems available for production and they’re called journaling filesystems. A journaling file system is a file system that logs changes to a journal (usually a circular log in a specially-allocated area) before actually writing them to the main file system.

A journaled file system maintains a journal of the changes it intends to make. In case of disruption, recovery involves reading the journal and replaying to restore consistency and ultimately making the recovery process much easier and more efficient. Journaling results in massively reduced time spent recovering a filesystem after a crash, and is therefore in high demand in environments where high availability is important, not only to improve recovery times on single machines but also to allow a crashed machine’s filesystem to be recovered on another machine when we have a cluster of nodes with a shared disk. Below is a list of a few major journaling filesystems that you can implement.

XFS

XFS is the oldest journaling file system available for UNIX systems, and has a mature, stable and well-debugged codebase. It a high-performance journaling file system. It provides quick recovery after a crash, fast transactions, high scalability, and excellent bandwidth. XFS provides journaling for file system metadata, where file system updates are first written to a serial journal before the actual disk blocks are updated. The journal is a circular buffer of disk blocks that is never read in normal filesystem operation.

It can be stored within the data section of the filesystem (an internal log), or on a separate device to minimise disk contention. On XFS the journal contains ‘logical’ entries that describe at a high level what operations are being performed, as opposed to other filesystems with ‘physical’ journals that store a copy of the blocks modified during each transaction. Journal updates are performed asynchronously to avoid incurring a performance penalty. In the event of a system crash, operations immediately prior to the crash can be redone using data in the journal, which allows XFS to guarantee file system consistency. Recovery is performed automatically at file system mount time, and the recovery speed is independent of the size of the file system. Where recently modified data has not been flushed to disk before a system crash, XFS ensures that any unwritten data blocks are zeroed on reboot, obviating any possible security issues arising from residual data.

ReiserFS

ReiserFS stores file metadata (“stat items”), directory entries (“directory items”), inode block lists (“indirect items”), and tails of files (“direct items”) in a single, combined B+ tree keyed by a universal object ID. Disk blocks allocated to nodes of the tree are “formatted internal blocks”. Blocks for leaf nodes (in which items are packed end-to-end) are “formatted leaf blocks”. All other blocks are “unformatted blocks” containing file contents. Directory items with too many entries or indirect items which are too long to fit into a node spill over into the right leaf neighbour. Block allocation is tracked by free space bitmaps in fixed locations.

By contrast, ext2 and other Berkeley FFS-like file systems simply use a fixed formula for computing inode locations, hence limiting the number of files they may contain.[9] Most such file systems also store directories as simple lists of entries, which makes directory lookups and updates linear time operations and degrades performance on very large directories. The single B+ tree design in ReiserFS avoids both of these problems due to better scalability properties.

JFS

IBM’s JFS is a journaling filesystem used in its enterprise servers. It was designed for “high-throughput server environments, key to running intranet and other high-performance e-business file servers” according to IBM’s Web site.

EXT3

Ext3 adds journaling filesystem capabilities to ext2fs. An alternative for all those who do not want to switch their filesystem, but require journaling capabilities. It is distributed in the form of a kernel patch and provides full backward compatibility. It also allows the conversion of an ext2fs partition without reformatting and a reverse conversion to ext2fs, if desired.

Resources:

* Linux XFS: http://oss.sgi.com/projects/xfs
* ReiserFS: http://www.namesys.com
* JFS for Linux: http://oss.software.ibm.com/jfs
* Ext3: http://www.zipworld.com.au/~akpm/linux/ext3

References:

* http://en.wikipedia.org/wiki/Journaling_file_system
* http://en.wikipedia.org/wiki/XFS
* http://en.wikipedia.org/wiki/Ext3
* http://en.wikipedia.org/wiki/IBM_Journaled_File_System_2_%28JFS2%29
* http://en.wikipedia.org/wiki/ReiserFS

Categories: linux, tools Tags:

Top 20 Linux Filesystem Tools

January 14th, 2010 madiga No comments

The Linux filesystem is complex and the first thing that most new users shifting from Windows will find confusing is navigating the Linux filesystem.

The second thing is that not too many people are familiar with how the filesystem works or know how to troubleshoot if any problems arise.

The good thing is, there are many different types of file system tools and utilities available to make troubleshooting easier but again not too many people are familiar with them. We’ve put together top 20 of the most useful file systems tools and utilities that you’ll find useful in your Linux environment.

  1. XFS - a high-performance journaling file system. It provides quick recovery after a crash, fast transactions, high scalability, and excellent bandwidth.

  2. e2salvage – a utility which tries to recover a data from damaged ext2 filesystems. Unlike e2fsck, it does not look for the data at particular places and it don’t tend to believe the data it finds; thus it can handle much more damaged filesystem. Moreover, fsck connects the found i-nodes to lost+found directory. e2salvage instead tries to recover the directory structure.

  3. e2fsprogs – contains essential ext2 filesystem tilities which consists of e2fsck, mke2fs, debugfs, dumpe2fs, tune2fs, and most of the ther core ext2 filesystem utilities.

  4. GNU Parted – a package for creating, destroying, resizing, checking, and copying partitions and the file systems on them. This is useful for creating space for new operating systems, reorganizing disk usage, copying data between hard disks, and disk imaging.

  5. Partition Image – a Linux/UNIX utility similar to Symantec’s Ghost. This uility saves partitions in the EXT2, Reiserfs, NTFS, HPFS, FAT16, and FAT32 file system formats to an image file. The image file can be compressed with gzip or bzip2 in order to save disk space, and it can be split in order to fit onto a series of floppy disks. This program can be useful for backup purposes. A boot/root disk is also provided, allowing you to run Partition Image without Linux installed on the hard disk.

  6. reiserfs - revolutionary new approach to file system design which stores not just filenames but the files themselves in a B*-tree. It is a generation ahead of alternatives which use older plain B-tree technology, and cannot store the files themselves in the tree. Reiserfs doesn’t suffer from log congestion either, you can effectively use it for quickly creating a 100,000 entry directory, and it is fairly unique in that.

  7. Filesystem in Userspace – a simple interface for userspace programs to export a virtual filesystem to the Linux kernel. It also aims to provide a secure method for non privileged users to create and mount their own filesystem implementations.

  8. LUFS Userland Filesystem – a hybrid userspace filesystem framework supporting many “exotic” filesystems (localfs, sshfs, ftpfs, httpfs, socketfs, freenetfs, and nutellafs) transparently for any application. It can be regarded as doing the same job as the VFS (virtual filesystem switch) in the kernel: it is a switch, distributing the filesystem calls to its supported filesystems.

  9. EncFS – an encrypted pass-through filesystem which runs in userspace on Linux (using the FUSE kernel module). Similar in design to CFS and other pass-through filesystems, all data is encrypted and stored in the underlying filesystem. Unlike loopback filesystems, there is no predetermined or pre-allocated filesystem size.

  10. OpenAFS – a distributed filesystem which offers a client-server architecture, transparent data migration abilities, scalability, a single namespace, and integrated ancillary subsystems.

  11. IOzone – filesystem benchmark tool. The benchmark generates and measures a variety of file operations. Iozone has been ported to many machines and runs under many operating systems.

  12. FSlint - a toolkit to find various forms of lint on a filesystem. At the moment it reports duplicate files, bad symbolic links, troublesome file names, empty directories, non stripped executables, temporary files, duplicate/conflicting (binary) names, and unused ext2 directory blocks.

  13. SSH Filesystem – SSH Filesystem uses the SSH File Transfer Protocol (SFTP), which is supported by most SSH servers. It is based on Filesystem in Userspace (FUSE), and hence root privilege is not required for mounting a remote filesystem. No setup is necessary and it is very easy to use.

  14. Dump/Restore – The dump package contains both dump and restore. Dump examines files in a filesystem, determines which ones need to be backed up, and copies those files to a specified disk, tape or other storage medium. The restore command performs the inverse function of dump; it can restore a full backup of a filesystem. Subsequent incremental backups can then be layered on top of the full backup. Single files and directory subtrees may also be restored from full or partial backups.

  15. Squashfs – a highly compressed read-only filesystem for Linux 2.4 and Linux 2.6. It uses zlib to compress files, inodes, and directories. All blocks are packed to minimise the data overhead, and block sizes of between 4K and 32K are supported. It is intended to be used as a filesystem for archival use and in embedded systems where low overhead is needed, and has been tested on PowerPC, i586, SPARC, and ARM architectures.

  16. Filelight – Filelight graphically represents a file system as a set of concentric segmented rings (or nested piecharts), aiming to show where disk space is being used.

  17. e2undel – an interactive console tool that recovers the data of deleted files on an ext2 file system under Linux. A library that allows you to recover deleted files by name is included. e2undel does not manipulate any internal ext2 structures, and it does not require any additional tools.

  18. SFS - a secure, global network file system with completely decentralized control. SFS lets you access your files from anywhere and share them with anyone, anywhere. Anyone can set up an SFS server, and any user can access any server from any client.

  19. Injector Linux – a single-floppy Linux system, which supports many filesystems — currently approximately 25. Because of its wide range of supported filesystems, it is very useful for disk rescue operations.

  20. File system investigator – a forensic tool for viewing filesystems. It is written entirely in Java, so it is platform independent. It allows viewing of ReiserFS and EXT2/3. While primarily intended as a forensics tool, it is also useful for Linux users who dual boot with another OS; it allows them to view and extract files from their partitions.

In conclusion, you don’t have to limit yourself to default tools available. Experiment with the applications mentioned above, stay ahead of the game by understanding how Linux filesystem is structured and utilized by users and applications.

Categories: tools Tags:

20 Tools for Linux to share with Windows

January 11th, 2010 madiga No comments

The gap between Linux and Windows filesystem has been improved alot. There are many different ways of seeing your windows partition on Linux. In this article, I’m going to list the 20 different types of useful applications you can use.  Some are simple and easy and some are a bit more complex but with great amount of extra features and such.

  1. Linux NTFS – provides Linux kernel drivers, a multiplatform NTFS library, and tools to create, resize, clone, rescue, query, label and fix NTFS volumes, and to undelete, resize, list, and query files for the filesystem used by Windows XP, 2003, 2000, NT4, and Vista. It also provides support for the Logical Disk Manager (LDM) that controls Windows’ Dynamic Disks and is used to create software mirrors, stripes, and RAID.

  2. SMB Web Client – a simple PHP script that allows users to access Windows networks from a Web browser (using Samba tools).

  3. Xfsamba – an SMB (Windows) network navigator with upload, download, rm, rmdir, mkdir, and tar capabilities. Xfsamba does not need to mount remote Windows shares. Drag and drop is enabled for file uploads/downloads.

  4. Smb4k – a SMB and CIFS (Windows) share browser for KDE. It uses the Samba software suite to access the SMB and CIFS shares of the local network neighborhood. Its purpose is to provide a program that’s easy to use and has as many features as possible.

  5. SMBNetFS – a user-space filesystem for Linux and FreeBSD that allows you browse a Samba/Microsoft network much like the network neighborhood in Microsoft Windows.

  6. Samba TNG – another SMB server for Unix-like operating systems. This software is concentrating on MSRPC interoperatability with Windows NT, 2000, XP, and 2003. It provides PDC support and tools to remotely administer NT servers and workstations. Note that Samba TNG is currently in a beta stage. If you need advanced file/print serving capabilities, you should opt for Samba classic.

  7. Davenport – a Java Servlet-based application providing WebDAV access to an entire SMB network. This enables users to connect to Windows and Samba shares via WebDAV clients such as Mac OS X, Windows Web Folders, etc. or any Web browser. Workgroups are browseable, and users are authenticated against a domain.

  8. IntegraTUM WebDisk – a Web application which gives you direct access to a file server. It is written using Java Servlets and the jCIFS library. Supported file servers are Samba, MS Windows, NetApp OnTAP, and any other server based on the CIFS protocol.

  9. Captive – provides full read/write access to NTFS disk drives in the WINE way by using the original Microsoft Windows ntfs.sys driver. It emulates the required subsystems of the Microsoft Windows kernel by reusing one of the original ntoskrnl.exe, ReactOS parts, or this project’s own reimplementations on a case by case basis. Involvement of the original driver files was chosen to achieve the best and unprecedented filesystem compatibility and safety.

  10. Sharity – mounts shares exported by Windows, Samba, and other SMB/CIFS servers in the file system of Unix computers. It implements Resource Browsing, which is similar to the Windows Network Neighborhood (Netbios Workgroups and Active Directory), NTLM, NTLMv2, and Kerberos authentication, Microsoft’s Distributed File System (DFS), and manipulation of Access Control Lists (ACLs).

  11. chntpw – a Linux utility to (re)set the password of any user that has a valid (local) account on your WinNT or Win2000 system, by modifying the crypted password in the registry’s SAM file. You do not need to know the old password to set a new one. It works offline (i.e., you have to shutdown your computer and boot off a linux floppy disk). The bootdisk includes stuff to access NTFS partitions and scripts to glue the whole thing together. This utility works with SYSKEY and includes the option to turn it off. A bootdisk image is provided.

  12. rfstool – allows you to access ReiserFS partitions from a Windows 95/98/ME/NT/2000/XP system. It also allows you to access ReiserFS partitions from Linux. It is a complete rewrite of the ReiserFS functions needed to list directories, copy files, and backup metadata.

  13. Scrounge NTFS – a data recovery program for NTFS filesystems. It reads each block of a hard disk and rebuilds the filesystem tree on another partition.

  14. Paragon NTFS for Linux – The Paragon NTFS for Linux driver provides read and write access to NTFS volumes for all kinds of files including system files.

  15. fatresize – a command line tool for non-destructive resizing of FAT16/FAT32 file systems. It is based on the GNU Parted library. The main target of the project is to be used with the EVMS FAT plugin.

  16. FUR filesystem – a filesystem based on FUSE which mounts a Windows CE device (connected with the librapi2 from the synce project) onto a directory of the local filesystem in a transparent and user-friendly way.

  17. WinShares – a Linux program that scans the network and mounts all Windows and Samba shares available, allowing the user to browse them. It uses CIFS.

  18. Explore2fs – a Windows program that provides read support for Linux ext2 and ext3 filesystems under NT4, Win95, Win98, WinME, Win2000, and WinXP. This is a Windows to Linux connector and it’s worth mentioning in this article.

  19. EXT2IFS - an Installable File System for Windows NT4, Windows 2000, and Windows XP. The driver can read both the EXT2 and EXT3 filesystems. A simple installation program makes using the driver easy. This is also a Windows to Linux connector and it’s worth mentioning in this article.

  20. NTFS-3G – a read/write NTFS driver. NTFS-3G is available for over 60 Linux distributions, including most major ones.

Lots of great tools to try…so little time.

Categories: linux, tools Tags:

Rsync backup helpers

January 6th, 2010 madiga No comments

To setup a quick and efficient backup system, all you need is rsync and that’s it. Rsync is a very powerful tool that can do anything and everything that has to do with moving files around within and across different networks and securely. With rsync doing backups and restores can be as quick as a few blinks of an eye!

Rsync has many features and techniques that can be utilized to setup the perfect backup/restore solution. In this article, we’re listing a few very useful tools that compliment rsync to ensure that you’ve used it to it’s best capacity.

  1. rsync-incr – a Linux wrapper shell (bash) script around rsync to perform automated, unattended, incremental, disk to disk backups, automatically removing old backups to make room for new ones. It produces standard mirror copies that are browsable and restorable without specific tools.

  2. RSync Manager – a set of Python scripts to add and update packages for rsync-based mirror sites, and also create some nice HTML statistics from the logs. It is mainly written for large site management, with extensibility in mind.

  3. Warsync – server replication system mainly used to sync servers in LVS clusters. It is based on rsync over ssh and has native support for Debian package synchronization.

  4. Rsync Vault Manager – an archive manager that uses rsync to manage backups of multiple clients across multiple logical partitions.

  5. Rsync Incremental Backup Script – an incremental backup system written using PHP, rsync, ssh, and cp. It can backup local machines and other networked devices, and is designed to be highly configurable and informative for system administrators. It performs a large amount of error checking, and includes logging and email capabilities.

  6. Grsync – a GUI for rsync, the command line directory synchronization tool. While it can work with remote hosts, its focus is to synchronize local directories.

  7. Rsyncrypto – allows you to encrypt a file or a directory structure such that they can later be synchronized to another machine using rsync.

  8. Rsyncbackup – a handy tool for scheduled backups using rsync. rsyncbackup lets you easily setup multiple source folders and destinations, both locally, on your iPod or external firewire disk, or at a remote destination using ssh.

  9. Rsync-backup – automates the process of backing up multiple systems to one or more backup servers. It focuses on making the process secure. It encrypts the backup going across the wire, only ships changed data, runs the server as root to preserve permissions and ownership, keeps people from seeing each other’s backups, and doesn’t require the server to trust any files sent from the clients.

  10. Backup Buddy – a set of scripts which make versioned, disk-to-disk, backups via rsync easy. It allows the user to set up disk-to-disk backups that happen automatically.

  11. Zsync – an implementation of rsync over HTTP. It allows updating of files from a remote Web server without requiring a full download or a special remote server application.

  12. NasBackup – a solution for backing up desktop PCs, laptops, and servers to network disks. It is highly configurable and uses rsync to only transfer file differences over the network.

  13. Rsnapshot – a filesystem snapshot utility based on rsync. It makes it easy to make periodic snapshots of local machines, and remote machines over ssh. It uses hard links whenever possible, to greatly reduce the disk space required.

  14. OtheRSync – a program for the automatic, efficient, and safe synchronization of two machines over a network. It’s efficient (using compression and incremental changes), safe (uses ssh, is fail-safe, and makes backups of deleted files by default), uses parallel processing, and supports feedback through the commandline prompt.

  15. Rsynchelper – helps you use the mirroring program rsync. rsynchelper makes it easier for groups of servers to cooperate in mirroring. rsynchelper makes it easier to make your content available for others to mirror, makes it easier to mirror someone else’s content, and it automates the maintenance of an accurate list of who is maintaining which content.

If you’re new to rsync and are not sure of it’s capabilities, try out some of the tools listed above which are developed for the sole purpose of using rsync as a backup solution.

Categories: linux, tools Tags: