Archive

Posts Tagged ‘tools’

Super commands for Unix beginners and veterans!

January 8th, 2013 No comments

As a sysadmin of many years, I’m always fascinated by the world of *Unix and the untapped commands and tricks that are still there that I haven’t found out yet.  There are numerous ways to do something quicker, easier and more efficiently if a sysadmin knows where to look.

In this article, we’re listing the top most useful commands that any sysadmin should know and use when troubleshooting or in general working around their way in a Linux or Unix environments in general.

This command allows you to save a file you edited in VI without the needed permission for that file:

$ :w !sudo tee %

An alternative to traceroute or ping command on Linux is to run mtr:

$ mtr yahoo.com

Ever wanted to execute a command without saving it in history?  Here we go:

$ <space>who  (This will run the ‘who’ command without it being recommended in your history file).  Set your PS1=” ” to do it automatically by prepending space.

Ever found your screen or garbled up and you end up closing session and logging back in?  Well dont’ do that! Just type:

$ reset

To capture your activity on Linux desktop just type:

$ ffmpeg -f x11grab -s wxga -r 25 -i :0.0 -sameq /tmp/wow.mpg

Want to reverse tunnel?  Here is how you start a tunnel from some external site’s port 80 to your local server port 8001.  You can then hit that site via http://localhost:8001

$ ssh -N -L2001:localhost:80 blah.com

Just a single job around 8PM, better than a cron:

$ echo “who;ps -eaf” | at 8pm

Now, for one of the most helpful if not the most helpful command that’ll allow you to bind port to a process ID…perhaps better than lsof:

$ netstat -tlnp

Ever wanted to tweet from shell?  Perhaps you want to alert your twitter that your service is down?

$ curl -u user:pass -d status=”I am tweeting nigga!” http://twitter.com/statuses/update.xml

Run your website all in memory, here is how:

$ mount -t tmpfs tmpfs /data/blah.com -o size=10g  (Assuming you have at least 10g on server to spare, you now have a new partition to which you can copy your website to)

How about a secure mount via SSH?  First download SSHFS and then:

$ sshfs user@server:/path/to/mount/from /path/to/mount/point

 

There is more!

Categories: linux Tags: , , ,

Tool of the week: urlShort

January 14th, 2011 No comments

Almost everyone and their Mama is bringing up a URL shortening web site. So we figured why not preview a neat tool that does the exact thing without any manual coding intervention.

urlShort is a URL shortener, complete with a full API, URL previews, simple installation and management, custom names/aliases, and more.

  • Restore reliability and transparency to short URLs by operating your own service specific to your domain.
  • Provide your own short URLs to your content, rather than relying on others.
  • Using PHP & MySQL, urlShort lets you provide short URLs similar to TinyURL, except on your domain.
  • Be smart: Offer users of social networks likeTwitter with reliable, and safe short URLs.
  • If you know PHP and MySQL, you can change it to do anything you want.
  • Feature complete, with a full API, custom names, simple installation and management, preview functionality, statistics and more.
  • Download it here.
Categories: tools Tags:

10 password management tools for Linux

December 16th, 2010 No comments

With so many different accounts and passwords, it’s much harder now to keep all those credentials in our heads.  Luckily there are tons of secure password management tools out there that we can use to keep our accounts safe and encrypted.  In this list, we’ve put together a list of 10.

cpm – a small console tool to manage passwords and other sensitive data and store them in a public-key encrypted file. It also allows you to configure the whole hierarchy yourself, so it’s easily adoptable for many requirements. The encryption is handled through GnuPG, and the data inside is stored as XML.

Strip – a password and account management program for the Palm Computing Platform. Designed to fit the needs of both IT professionals and the average user, it combines ease of use with flexibility and security. 128 bit encryption ensures that even if your Palm Pilot is lost or stolen your important account and password information will remain confidential. Strip has a quick and easy to use interface, with many useful features including the ability to beam shared accounts to other Strip users.

phpPasswordManager – a Web-based password management system. It organizes your different accounts in groups and makes them accessible over a Web interface. Each account can be encrypted with a different master password.

Pwman3 – a console based password management application. It uses SQLite to store information, and can encrypt it with AES, ARC2, ARC4, Blowfish (default), CAST, DES, DES3, IDEA, or RC5.

Open Sesame – a Web application for password management.

Locked Area – a highly sophisticated password protection and membership management system. It has been designed to be as secure as possible while it still runs hands-free with no input from the Webmaster needed. Locked Area uses Apache’s .htaccess and .htpasswd along with DES randomized salt or MD5 encryption of passwords for increased security. It also includes a member database that lets the administrator maintain a mailing list along with the member’s area.

Secret Squirrel – a basic password management application written in Java. It uses Blowfish and SHA-384, performs password generation using the Java secure random object (fully configurable), and supports password groups.

myPMS – a Web-based password management application which will require you to remember only one password, making the management of your passwords simple. It uses simple HTML, PHP, and a MySQL database to store your passwords.

Password Chest – a simple password management application. It reads and writes the Password Safe V3 format and is meant to provide a native Mac OS X user experience.

Web-KeePass – a Web based port of the KeePass project, which is an easy-to-use password manager which helps you to manage your passwords in a secure way. All passwords are unlocked with one master password. It is focused on deep encryption, password secrecy, and ease of use. Web-KeePass is a full-featured, client-server application. It comes bundled with Tomcat 5.5 but is easily configured to any Java Servlet Web server.

Categories: linux, tools Tags: ,

15 Note Taking Applications for Linux

December 12th, 2010 6 comments

Nothing beats the traditional methods of note taking, a simple pen and paper.  Nowadays it’s very common for many of us to use an application such as notepad, Evernote or some other note taking tool to jot down and organize our thoughts and work.

In this article, we’re listing top 15 populate note taking applications for Linux that are simply to the point.

  1. nts – an acronym for Note Taking Simplified. It provides a simple, intuitive format for using plain text files to store notes, a command line interface for viewing notes in a variety of convenient ways, and a cross-platform, wx(python)-based GUI for creating and modifying notes as well as viewing them. Displayed items can be grouped by path or tag and can be filtered in various ways.
  2. MyNotex – software to take and to manage textual notes, even large numbers of them. Notes are gathered under different subjects and consist of a title, a date, a list of tags (keywords), and a free-length text which can be formatted (change font color, name, size, italic, bold, etc.); the software can manage bullets and numbered lists, although without automatic indentation. A single file produced by MyNotex contains multiple subjects and notes.
  3. Nodau – a simple console based note taking program, it allows you to easily create, edit, and view notes, and search them by name or date.
  4. tnote – a small note taking program for the terminal. It aims to be small, quick, and easy to use, and is similar to sticky style GUI note taking apps such as Tomboy notes.
  5. Notalon – a simplified note-taking application. Its aim is to streamline your focus on taking notes on books and during class lectures.
  6. Mediabird – allows students to document and review their study effort using a collaborative notepad. The Web2.0 application allows for taking notes on a personal notepad. A virtual text marker allows the students to highlight their notes and ask questions about the highlighted parts.
  7. Nurpawiki – a personal information manager (PIM) application that combines a wiki, a to-do list, and a simple scheduler to help you get organized. It aims to ease note taking and action planning. Actions (to-dos) are always associated with notes or plans (wiki pages). The application will be extended for multiple users in the future to help organizing small team development.
  8. NoteFinder – an application for managing any text information: notes, quotes, conversations, drafts, articles, contacts, etc.
  9. Tomboy – a desktop note-taking application for Linux and Unix. It is simple and easy to use, but has the potential to help you organize the ideas and information you deal with every day.
  10. Web-NoteTaker – a PHP/MySQL Web-based note taking tool designed for “formal” note taking. It was originally designed to facilitate the author’s note taking process, which required tracking who said what during a meeting and then how what was said “evolved”. The tool allows you to attribute notes to attendees and give notes “types”, and it timestamps the entries. At this point the tool does not allow editing through the UI, but does allow entry of notes through the UI.
  11. gnoteman – a quick and simple note taking applet for the GNOME panel. It has a bare minimum of features, and is primarily used to jot down a quick note for future reference. It can also be used as a “sticky notes” applet.
  12. Tomboy-LaTeX – a plugin for the desktop note-taking application Tomboy. It automatically converts LaTeX math code enclosed in \[…\] into inline images. Clicking on the image and moving the cursor near the image reveals the markup again.
  13. Noty – a small note-taking application. It allows for small notes with formatting that can be organized into a tree-like structure and which also use internal hyperlinks.
  14. The Note-Taking Tool – a note-taking application tailored for use by students in academic environments. It makes use of the well accepted Cornell Note-Taking method.
  15. Thumbpad – a simple note taking program, which displays a very large keyboard that you can easily type on using only one finger. So, if you’re holding your PDA with one hand, you can use your thumb.
Categories: tools Tags: ,

Top 20 Most Popular Open Source Search and Indexing Tools!

December 10th, 2010 No comments

When a website gets too big, with thousands and millions of documents, it gets harder and harder to manage the entire thing. Some people to keep things organized buy expensive commercial products to regularly index their documents for quick search but there are also some who take advantage of what’s available for free and open source.

In this article, we’ve put together a list of top 20 most popular open source search and indexing packages that you can use on your website for just that and much more.

  1. Xapian – a search engine library, scalable to collections containing hundreds of millions of documents. It’s written in C++ with bindings for Perl, Python, PHP, Java, Tcl, C#, and Ruby. It is a highly adaptable toolkit that allows developers to easily add advanced indexing and search facilities to their own applications. It supports the Probabilistic Information Retrieval model and also a rich set of boolean query operators. Omega is a Web search application built upon the Xapian library. It can index a Web server’s document tree (including HTML, PDF, OpenOffice, MS Word/Excel/Powerpoint/Works, WordPerfect, RTF, PS, etc.), or data exported from arbitrary sources (e.g. SQL databases).
  2. Open Search Server – stable, high-performance search engine and a suite of high-powered full text search algorithms. Documents can be indexed in sixteen languages. Multi-lingual analyzers slice sentences into words, then run lemmatisation algorithms on words based on the document’s language.
  3. Splunk – an IT search engine that indexes and lets you search, navigate, alert, and report on data from any application, server, or network device. It lets you securely access logs, configurations, scripts and code, messages, traps and alerts, activity reports, stack traces, and metrics across thousands of components from one place in real time.  Not exactly open source but freeware version is available.
  4. LinPHA – an easy-to-use, multilingual, flexible photo/image archive/album/gallery written in PHP. It uses an SQL database (MySQL/PostgreSQL/SQLite) to store information about your pictures. It comes with an HTML-based installer, so you don’t need experience in setting up SQL databases.
  5. PhpDig – a Web spider and search engine written in PHP, using a MySQL database and flat files. It builds a glossary with words found in indexed static and dynamic pages. On a search query, it displays a result page containing the search keys, ranked by occurrence. It includes a template system and can index PDF, Word, Excel, and PowerPoint documents using external tools.
  6. Urd – a Web-based Usenet binary download manager. It stores the newsgroup information in a MySQL database and aggregates the articles into sets of a single download (e.g. one album or movie). The Web interface can be used to search with regular expressions. It uses its own downloading daemon that has support for scheduling downloads and updating databases.
  7. phpAutoindex – a directory listing generator meant as a replacement for the Apache module autoindex. phpAutoindex is more flexible and configurable than the Apache module. There are many input plugins that can be used to display information such as the size or the SHA1 hash of a file.
  8. TSEP – a simple, yet very powerful and fast PHP website search engine. It is built to index your site so it can be searched later within seconds. Some features: your own stopwords, logging, MySQL Boolean search, many languages, CSS formatting.
  9. Ex-Crawler – divided into three subprojects. The main part is the Ex-Crawler daemon server, a highly configurable and flexible Web crawler written in Java. It comes with its own socket server, with which you can manage the server, users, distributed grid/volunteer computing, and much more. Crawled information is stored in a database (Currently MySQL, PostgreSQL, and MSSQL are supported). The second part is a graphical (Java Swing) distributed grid/volunteer computing client, including user computer state detection, based on JADIF Project. The Web search engine is written in PHP. It comes with a Content Management System, user language detection and multi-language support, and templates using Smarty, including an application framework that is partly forked from Joomla 1.5, so that Joomla components can be adapted quickly.
  10. LargeXMLSitemap – allows PHP developers to generate search engine XML sitemaps with millions of URLs. It automatically handles correct sizing, splitting, and compression of the generated XML file structure. Unlike other tools, LargeXMLSitemap does not crawl your site to generate an XML structure of your browseable URLs; it’s a collection of PHP classes that can be used from within your site’s code to generate these structures.
  11. X search engine – a Web-based search engine that is easy to install. A ready-to-use database with 30,000 records is included. It is only 2MB in size.
  12. focuseek searchbox – a family of easily installable full-text search engines that can spider Internet and intranet data sources (Web sites, newsgroups, FTP sites, and others) or index data you feed to it and make it available for searching.
  13. xinabse – a search engine for small to medium-sized sites. It consists of a HTTP spider written in Perl and a templatable frontend in PHP. Keywords and sites are stored in a MySQL database.
  14. OpenWebSpider – a multi-threaded Web spider (also known as a robot or crawler) and search engine with a lot of interesting features.
  15. The PKP Open Archives Harvester – a free metadata indexing system intended to expand and improve access to research. The PKP OAI Harvester allows you to create a searchable index of the metadata from Open Archives Initiative (OAI)-compliant archives, such as sites using Open Journal Systems (OJS) or Open Conference Systems (OCS).
  16. CrawlTrack – a PHP script which makes it possible to follow the visits of search-engines crawlers and spiders on your Web site. Data are displayed in the form of tables and graphics which provide very legible information.
  17. HITWEB – a Web site which provides people with a collection of thoroughly ranked Internet sites. Unlike a Web search engine, HITWEB requires human intervention to build its collection of links. HITWEB uses PHP, PHPLib (for templates), and MySQL.
  18. Larbin – Web crawler intended to fetch a large number of Web pages to fill the database of a search engine. With a network fast enough, it should be able to fetch more than 100 millions pages on a standard PC. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin.
  19. Basilic – a bibliography server for research laboratories. It automates and facilitates the diffusion of research publications over the Internet, automatically generating Web pages from a publication database. Each publication has an associated Web page, which provides downloads and additional documents (abstract, images, BibTeX). Index pages are also created, including a search engine with several options for results display. New publications can be added to the database in an instant.
  20. mnoGoSearch – a full-featured web search engine software for intranet and internet servers.
Categories: tools Tags:

Top 30 Most Popular Scientific and Engineering Packages for Linux

December 8th, 2010 No comments

In this article, I’ve come across 30 of the most popular Scientific and Engineering free and open source packages that students, scientists and programmers in the engineering field can greatly appreciate.  These packages aren’t categorized in any particular way, it’s just a top 30 list of packages that has been most popular lately.

  1. GNU TeXmacs – a free wysiwyw (what you see is what you want) editing platform with special features for scientists. The software aims to provide a unified and user friendly framework for editing structured documents with different types of content: text, mathematics, graphics, interactive content. TeXmacs can also be used as an interface to many external systems for computer algebra, numerical analysis, and statistics.
  2. white_dune – a graphical VRML97/X3DV editor, simple NURBS/Superformula 3D modeller, animation tool, and a very limited VRML97/X3DV commandline compiler in development. VRML97 (Virtual Reality Modeling Language) is the ISO standard for displaying 3D data over the Web via browser plugins (“HTML for realtime 3D”).
  3. Mathomatic – a portable, general-purpose computer algebra system (CAS) that can solve, differentiate, simplify, combine, and compare algebraic equations, perform standard, complex number, and polynomial arithmetic, etc. It does some calculus and is very easy to compile, learn, and use.
  4. BRL-CAD – a powerful constructive solid geometry solid modeling system that includes an interactive geometry editor, ray-tracing support for rendering and geometric analysis, path-tracing for realistic image synthesis, network distributed framebuffer support, and image and signal-processing tools.
  5. nut – nutrition software to record what you eat and analyze your meals for nutrient composition. The database included is the latest USDA Nutrient Database for Standard Reference. This database contains values for vitamins, minerals, fats, calories, protein, carbohydrates, fiber, etc., and includes the essential polyunsaturated fats, Omega-3 and Omega-6. Nutrient levels are expressed as a percentage of the Daily Value, the familiar standard of food labeling in the United States, but also can be fully customized.
  6. Asymptote – a powerful descriptive 2D and 3D vector graphics language for technical drawing, inspired by MetaPost but with an improved C++-like syntax. It provides for figures the same high-quality level of typesetting that LaTeX does for scientific text. Asymptote is a programming language as opposed to just a graphics program. It can exploit the best features of script (command-driven) and graphical user interface (GUI) methods. High-level graphics commands are implemented in the language itself, allowing them to be easily tailored to specific applications.
  7. Blender – a free 3D animation studio. It includes tools for modeling, sculpting, texturing (painting, node-based shader materials, or UV mapped), UV mapping, rigging and constraints, weight painting, particle systems, simulation (fluids, physics, and soft body dynamics and an external crowd simulator), rendering, node-based compositing, and non linear video editing, as well as an integrated game engine for real-time interactive 3D, and game creation and playback with cross-platform compatibility.
  8. Viking – GTK2-based GPS data editor and viewer. It can download and show Terraserver maps, import and draw tracks and waypoints over them, add coordinate lines, make new tracks and waypoints, hide different things, and more.
  9. Gpsdrive – a map-based navigation system. It displays your position on a zoomable map provided from a NMEA-capable GPS receiver. The maps are autoselected for the best resolution, depending of your position, and the displayed image can be zoomed. Maps can be downloaded from the Internet with one mouse click. The program provides information about speed, direction, bearing, arrival time, actual position, and target position.
  10. graphviz – a set of graph drawing tools and libraries. It supports hierarchical and mass-spring drawings; although the tools are scalable, their emphasis is on making very good drawings of reasonably-sized graphs. Package components include batch layout filters and interactive editors for X11, Java, and a TCL/tk extension. The batch filters can be configured as a web visualization service (using GIF and click-maps). Typical applications include display of finite state machines, software diagrams, database schemas, and communication networks.
  11. POV-Ray – a high-quality tool for creating 3D graphics. It is available in official versions for Windows, Mac OS/Mac OS X, and i86 Linux.
  12. R – a language and environment for statistical computing and graphics. It is similar to S, which was developed at Bell Laboratories by John Chambers et al. It provides a wide variety of statistical and graphical techniques (linear and nonlinear modelling, statistical tests, time series analysis, classification, clustering, etc.). R is designed as a true computer language with control-flow constructions for iteration and alternation, and it allows users to add additional functionality by defining new functions. For computationally intensive tasks, Fortran and C code can be linked and called at run time.
  13. GRASS – a software raster- and vector-based GIS (Geographic Information System), image processing system, graphics production system, and spatial modeling system. It contains many modules for raster data manipulation, vector data manipulation, rendering images on the monitor or paper, multispectral image geocoding and processing, point data management and general data management. It also has tools for interfacing with digitizers, scanners, and the PostgreSQL, DBF, and ODBC connected databases. GRASS operates on all common operating systems.
  14. Frink – a calculating tool and programming language designed to help you in the real world. It tracks units of measurement throughout all calculations and ensures that answers are correct. It converts between systems of measurement, and has a huge library of physical data. It is both a simple calculator for quick calculations and a full-fledged programming language for large tasks.
  15. QCad – a powerful but easy to use 2D CAD program for Linux, Windows, and Mac OS X. It uses DXF as its standard file format. While other CAD packages are often rather complicated to use, QCad tries to stay comfortable, and even an absolute beginner can create professional drawings with a minimum of effort.
  16. Hugin – a toolkit for stitching photographs and assembling panoramas, together with an easy to use graphical front end.
  17. GeoServer – a software server that allows users to share and edit geospatial data. Designed for interoperability, it publishes data from any major spatial data source using open standards. GeoServer is the reference implementation of the Open Geospatial Consortium (OGC) Web Feature Service (WFS) and Web Coverage Service (WCS) standards, as well as a high performance certified compliant Web Map Service (WMS). GeoServer forms a core component of the Geospatial Web.
  18. Freeverb3 – library includes high quality freeverb, nreverb, impulse response reverb, FIR filter constructor, band splitter, and hardknee/softknee compressor. Its oversampling feature provides high quality audio processing. The compressor provides mastering quality audio processing. This library includes XMMS plugins, audacious plugins, BMP beep media player plugins, and sample programs, which enable you to test the audio effects easily.
  19. GLE – a graphics scripting language designed for creating publication quality figures (e.g., a chart, plot, graph, or diagram). GLE supports various chart types (including function plot, histogram, bar chart, scatter plot, contour plot, color map, and surface plot) through a simple but flexible set of graphing commands.
  20. Stellarium – renders 3D photo-realistic skies in real time with OpenGL. It displays stars, constellations, planets, nebulas and others things like ground, landscape, fog, etc.
  21. Gpredict – a real time satellite tracking and orbit prediction program. Besides the general orbital data for satellites, gpredict can also calculate the footprint, visibility, doppler shift, signal loss, and signal delay for each satellite relative to one or more ground stations. The calculated data can be viewed in tables, on maps, or on polar graphs.
  22. TAMS – a qualitative or ethnographic coding and data extraction-analysis system.
  23. gEDA – suite of electronic design automation (EDA) tools. Currently, the project offers a mature suite of free software applications for electronics design, including schematic capture, attribute management, bill of materials (BOM) generation, netlisting into over 20 netlist formats, analog and digital simulation, and printed circuit board (PCB) layout.
  24. Visual Paradigm for UML – design tool that supports all UML diagrams, SysML diagrams, and entity relationship diagrams. Visual Paradigm for UML provides extensive use case modeling features, including full function UML use case diagram, flow of events editor, use case/actor grid, and activity diagram generation. Visual Paradigm for UML produces system documentation in PDF, HTML, and MS Word formats.
  25. ATLAS – project is an ongoing research effort focusing on applying empirical techniques in order to provide portable performance.
  26. Social Networks Visualizer – a flexible and user-friendly tool for the analysis and visualization of Social Networks. It lets you construct mathematical graphs with a few clicks on a virtual canvas, load networks of various formats (GraphViz, GraphML, Adjacency, Pajek, UCINET, etc), or create a network by crawling all links in a Web page.
  27. Visifire – a set of data visualization components powered by Microsoft Silverlight. It lets you create and embed visually stunning animated Silverlight Charts within minutes. Visifire is easy to use and independent of the server side technology. It can be used with ASP, ASP.Net, PHP, JSP, ColdFusion, Ruby on Rails, or just simple HTML.
  28. gpsd – a daemon that listens to a GPS or Loran receiver and translates the positional data into a simplified format that can be more easily used by other programs, like chart plotters. The package comes with a sample client that plots the location of the currently visible GPS satellites (if available) and a speedometer.
  29. Quantum GIS – a cross-platform Geographic Information System (GIS). It offers support for vector and raster formats, including spatially enabled tables in PostgreSQL using PostGIS, common GIS vector formats such as Shapefiles, and geo-referenced rasters (TIFF, PNG, and GEOTIFF). Many plugins are available to dynamically add new functionality. Viewing of GRASS layers (vector and raster) is provided by a plugin.
  30. Celestia – an OpenGL-based 3D space simulation for Unix and Win32 that lets you travel through the solar system, to the stars, and even beyond the galaxy. Visit over 100,000 stars, 100 solar system bodies, and all known extrasolar planets.
Categories: linux, tools Tags: ,

Tool of the week – iptstate

December 7th, 2010 2 comments

I recently came across this neat tool called IP Tables State or in short iptstate.  It basically works like top but displays the network stat in real-time.  It also has the ability to delete/remove an entry from the stat table.

This tool can be very helpful in troubleshooting or simply analyzing your stat table by sorting and filtering the table in several different ways such as by protocol or service.

Categories: tools Tags:

Tool of the week: MultiTail

November 29th, 2010 No comments

Want tail on steroids?  Try MultiTail, it creates multiple windows per tail, adds colors and other things that organizers  while you’re tailing your life away.

MultiTail lets you view one or multiple files like the original tail program. The difference is that it creates multiple windows on your console (with ncurses). It can also monitor wildcards: if another file matching the wildcard has a more recent modification date, it will automatically switch to that file. That way you can, for example, monitor a complete directory of files.

Merging of 2 or even more logfiles is possible. It can also use colors while displaying the logfiles (through regular expressions), for faster recognition of what is important and what not. It can also filter lines (again with regular expressions). It has interactive menus for editing given regular expressions and deleting and adding windows.

One can also have windows with the output of shell scripts and other software. When viewing the output of external software, MultiTail can mimic the functionality of tools like ‘watch’ and such.

Download it here.

Categories: tools Tags:

Tool of the week: ps-watcher, monitor your system processes

November 8th, 2010 No comments

In this week’s edition of tool of the week, we’re impressed enough to showcase ps-watcher as it’s a useful tool for any user or system administrator working on a Linux server to provide process level details.

This program runs the ps command periodically and triggers commands on matches. The match patterns are Perl regular expressions which can refer to the process information via variables.

For example it can be used to ensure that a daemon is running, or is not running too many times. It can also be used to determine when a process has consumed too many resources, perhaps due to a memory leak.

Categories: tools Tags:

command line alternatives to wget and so much better!

September 30th, 2010 5 comments

Most people including myself are hooked on using wget to do whatever quickies that we need to do on our servers. I use it in my scripts, crontab entries and even site mirroring and web crawling.

[ad]

If you’ve used it extensively, you’ve probably started to see it’s limitations when it comes to downloading large files in a short period of time, multi-threading, and web caching and web crawling.  Yes all that is possible with wget but it’s not powerful enough to stick to it.

The following is a list of alternatives that’ll blow wget away. All of the applications work on Unix/Linux, MacOS, Cygwin and probably everything else if you can get them compiled.

  1. I’ve come across a little program called Axel. It tries to accelerate downloads by using multiple connections for one download. It opens more than one HTTP/FTP connection per download and each connection transfers its own, separate, part of the file. This comes handy when some sites limit the speed of each connection so by opening more than one connection at a time multiplies the allowable bandwidth. Once all parts of the file is downloaded, it is seamlessly put together into original. This makes it better than wget and a more qualified tool for downloading large files quicker.
  2. The second tool that rocks is Aget. It is a multi-threaded download accelerator. It supports HTTP downloads and can be run from the console. From the Aget website: Tests show that Aget is successful in realizing its objectives. A file of size 36.347.010 bytes was downloaded in 14 minutes 28 secs via wget; whereas it was downloaded in 3 minutes and 15 seconds via Aget! Amazing indeed.
  3. The third super getter is Prozilla. A download accelerator for Linux. It makes multiple server connections to download a file, and also supports FTP searching for faster mirrors. Supports FTP & HTTP and the file will be downloaded as fast as possible as your bandwidth allows if not otherwise specified.
  4. Then there is Curl. An awesome command line tool for transferring of data and manipulating it in ways one couldn’t imagine.  Not enough can be said about Curl as it’s an awesome alternative to wget.
  5. The fifth rock star is Manda. Manda is a threaded download accelerator. It downloads files via HTTP by splitting the files into parts, speeding the download. I don’t have too much info on this as I didn’t get to test it but it’s in the same rank as axel and aget.
  6. The sixth punk star is GetFast. GetFast is a file download accelerator. It is multi-threaded and allows the downloading of Web pages and their sub-links. GetFast is an animal like the rest. It fetches anything and everything in lightening speed.
  7. How about giving Aria2 a try? A lightweight  multi-protocol & multi-source, cross platform download  utility operated in command-line  Use it as a bitTorrent client!
  8. And then there is Mulk. Not in the same league as above but it does the same job.  Multi-connection command line tool to download Internet sites. Similar to wget and cURL, but it manages up to 50 parallel links. Main features are: recursive fetching, Metalink retrieving, segmented download and image filtering by width and height.

Wget is an amazing tool with lots of options and is perfect for many things but if you’re looking for speed and getting large things done in small time, then give these apps a try. And the best part is, unlike wget, these apps resume from where it ended in case download process gets interrupted.

[ad]

I am probably missing other download accelerators that are on the same rank as the ones above so please put it up in comment area and I will update the article to include all.

Categories: linux, tools Tags: