jump to navigation

How to Backup Linux Content to Amazon S3 Using s3cmd October 21, 2012

Posted by tournasdimitrios1 in Linux, Linux admin tools.
2 comments

Amazon S3 provides a simple web services that can be used to store and retrieve any amount of data , at any time , from anywhere on the web . It gives any web-developer or administrator access to the same highly scalable , reliable , secure , fast , inexpensive (0.1$/GigaByte for the S3 service) infrastructure  that Amazon uses to run its own global network of web sites .Actually Amazon has a whole suite of services (IAAS , PAAS , SAAS ) on which admins and web-developers can rely upon . There are three ways to interact with these services :

  1. Web based AWS Management Console (your web-browser)
  2. Command line tools (interacting via terminal or scripts)
  3. An API for programmers (almost all programming languages have libraries to interact with )

From all available services ,  S3 is the simplest to use and targeted to a broad range of users . Possible uses could be : personal photo storage  , CDN for web-developers , media-library , for sysadmins (backing up their file-system) . This article will demonstrate a practical example of how this service can be used by Linux administrators  .  Just by installing a package , sysadmins (even with no programming knowledge)  can use their terminal or run a cron-job  to back-up / restore  critical file-system or application data .

S3cmd is a command line tool for uploading , retrieving and managing data into Amazon’s S3 Cloud storage service . It is best suited for power users who don’t fear command line . It is also ideal for scripts , automated backups triggered from cron , etc. This tool can be installed via your distributions package manager ( Yum , apt-get , homebrew –MacOs-) . Although I ‘ ll use CentOs to do the demonstration , the same concepts apply to all variants of the Linux operating system (even on MacOs) .
Before storing anything into S3 you must sign up for an “AWS” account (where AWS = Amazon Web Services) to obtain a pair of identifiers : Access Key and Secret Key. You will need to give these keys to S3cmd (only once during initial setup) . Think of them as if they were a username and password for your S3 account .

This article is targeted to Linux sysadmins , a future article will “satisfy”  web-developers . I ‘ll demonstrate how a PHP-library can be used to query Amazon’s S3 web-service (accomplishing identical functionality to s3cmd)

Prerequisites : The reader should already have an AWS  account , the process of registering for an developer account is simple . As aforementioned , using the terminal is a must , as S3cmd is a  command line back-up / restore tool . Writing Bash-scripts is only necessary if the administrator aims to automate some back-up tasks (cron jobs) . (more…)

Monitor Network Traffic by PID with “nethogs” on Linux March 9, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

Have you ever notice some suspicious spikes in your internet connection and wondered “What the hell is using up all that bandwidth ? ” , or worse , ” Have I been hacked?” . If so , then nethogs might be the program for you . It’s a lightweight  Linux terminal-based tool that monitors bandwidth usage , then groups it by process , so you can see which PID’s are using the most bandwidth  , and if necessary , kill them with gnome-system-monitor or kill .

If EPEL is on the repository list of CentOs 5.x ,  yum will handle the installation .

I’m using tmux and  kill to demonstrate nethogs . In this example somebody is downloading 400M from my web-server .

Similar utility is iftop .

sipcalc The Advanced subnet calculator on Linux March 7, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

Sipcalc is an advanced console-based IP sub-net calculator . It can take multiple forms of input (IPv4/IPv6/interface/host-name) and output a multitude of information about a given sub-net .In it’s simplest form takes an ip-address and a subnet mask on the command line and outputs information about the subnet. Sipcalc has support for both IPv4 and IPv6 addresses.

How to install sipcalc : (more…)

Passive Operating System Fingerprinting with p0f on Linux March 6, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

Passive fingerprinting works by quietly examining packets for patterns and not  sending data directly to a target host .Due to this passive analysis, the remote system will not be able to detect the packet capture.The process is completely passive and does not generate any suspicious network traffic. Although  other well-known and tested tools ( like nmap , ettercap , Siphon) exists , p0f is considered the granddaddy of passive operating system  fingerprinting . The O in operating system is replaced with a 0 (zero) character .

There are two methods of detecting the type of Operating System a host is running.

  • Active OS fingerprinting has been the most widely used method when analyzing a system. This is the method used in tools such as  nmap by Fyodor (http://www.insecure.org/nmap). This method includes sending crafted, abnormal packets to the remote host, and analyze the replies being returned from the remote host. Different TCP stacks will give different replies and thus allowing the analyzer tool to recognize a particular OS. If the remote host’s network is being protected by IDS or firewall devices, such attacks will be detected.
  • Passive OS fingerprinting on the other hand will not contact the remote host, but instead capture traffic coming from a connecting host going to the local network. The packets being captured are the ones the remote host sends when it attempts to establish a connection to a host on the local network.

Active OS fingerprinting is a fast process and a large number of hosts can be scanned in a short time frame. Passive fingerprinting on the other hand is a much slower process, and will work best if used on stored data (from a file) .

p0f can identify the system on machines that connect to your box, machines you connect to, and even machines that merely go through or near your box .

(more…)

ngrep — searching network packets like Unix grep March 5, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

ngrep strives to provide most of GNU grep’s common features, applying them to the network layer. ngrep is a pcap-aware tool that will allow you to specify extended regular expressions to match against data payloads of packets. It  currently recognizes TCP, UDP and ICMP across Ethernet, PPP, SLIP, FDDI and null interfaces, and understands bpf filter logic in the same fashion as more common packet sniffing tools, such as tcpdump and snoop .For a network administrator familiar with pattern matching with grep, ngrep requires a minimum of training .

On RedHat based distributions this utility can be installed through the yum installer , if rpmforge is on the repository list .

(more…)

Password Sniffing with “dsniff” on the Local Network March 4, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

The dsniff tool is a member of the Dsniff suit toolset , it’s an advanced password sniffer that recognizes several different protocols, including TELNET, FTP, SMTP, Post Office Protocol (POP), Internet Message Access Protocol (IMAP), HTTP, CVS, Citrix, Server Message Block (SMB), Oracle, and many others. Whereas other sniffers such as Wireshark will give you tons of additional information about the connection and the individual packets , you use dsniff if all you want are usernames and passwords.

The only  argument that dsniff can use is a tcpdump packet-filter expression so that you can specify what kind of traffic you want to sniff for passwords.

(more…)

Flood network with random MAC addresses with macof tool March 4, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

Macof is a member of the Dsniff suit toolset and mainly used to flood the switch on a local network with MAC addressess . The reason for this is that the switch regulates the flow of data between its ports. It actively monitors (cache)  the MAC address on each port, which helps it pass data only to its intended target. This is the main difference between a switch and passive hub. A passive hub has no mapping, and thus broadcasts line data to every port on the device. The data is typically rejected by all network cards, except the one it was intended for. However, in a hubbed network, sniffing data is very easy to accomplish by placing a network card into promiscuous mode. This allows that device to simply collect all the data passing through a hubbed network. While this is nice for a hacker, most networks use switches, which inherently restrict this activity.

Dsniffs “macof” generates random MAC addresses exhausting the switch’s memory. It is capable of generating 155,000 MAC entries on a switch per minute. Some switches than revert to acting like a hub.

(more…)

Reading pcap files with tcpshow on Linux March 4, 2011

Posted by tournasdimitrios1 in Linux admin tools.
add a comment

tcpshow reads a pcap file created from utilities like tcpdump , tshark , wireshark etc , and provides the headers in packets that match the boolean expression . The headers belonging to protocols like  Ethernet , IP , ICMP , UDP and TCP  are decoded . It represents an alternative to using tcpdump to decode data. The primary advantage of tcpshow is much nicer formatting for output. For example, here is the tcpdump  output for 3 packets:

(more…)

DNS spoofing with “dnsspoof” on Linux March 3, 2011

Posted by tournasdimitrios1 in Linux admin tools.
1 comment so far

Dnsspoof is a member of the Dsniff suit toolset and  works similarly to arpspoof . It lets you forge DNS responses for a DNS server on the local network . DNS runs on User Datagram Protocol (UDP), a connectionless protocol, a DNS client will send out a query and expect a response . The query is assigned a pseudo random identification number  which should be present in the answer from the DNS server. Then when the answer from the DNS server will be received , it will just have to compare both numbers if they’re the same, the answer is taken as valid, otherwise it will be simply ignored . The DNS protocol relies on UDP for requests (TCP is used only for zone transfers –> communications between DNS servers), which means that it is easy to send a packet coming from a fake IP since there are no SYN/ACK numbers (Unlike TCP, UDP doesn’t provide a minimum of protection against IP spoofing) . (more…)

Etherape the graphical network monitoring tool February 24, 2011

Posted by tournasdimitrios1 in Linux admin tools.
2 comments

Every Linux administrator needs a good network monitor for network management. But with so many to choose from it’s hard to know which one is best. For example a quick search at  Freshmeat.net returned 215 entries for open source network analyzers and network tools.My blog has already present some command line tools , and future articles will present even more , but it’s time to introduce  a graphical alternative . It’s just a matter of taste , some admins prefer the terminal tools (myself included )  and some others , the  graphical alternatives .

The EtherApe network monitor is a midrange option for monitoring your network’s data traffic. It can monitor your network cart directly , or read from a pcap file that was created from other utilities (tcpdump , wireshark , ettercap … ) . As an open source network monitor, EtherApe offers a dynamic graphical interface; features IP and TCP modes; supports Ethernet, FDDI, PPP, and slip devices; filters traffic; and reads traffic from both a tcpdump file and live from the network.

Installing etherape : (more…)

Follow

Get every new post delivered to your Inbox.

Join 53 other followers