GNU Linux/Tips

From WhyAskWhy.org Wiki
Jump to: navigation, search


Distribution Independent

Tip/Tricks/Guides that work for any distribution (or at least should).

Output Redirection

  • Save both standard output and error message: >&
  • Save standard output only: > or 1>
  • Save error messages only: 2>

These can be combined, like so:

 gcc test.c 2> error.msg > /dev/null

This saves error messages but tosses standard output

Background and Foreground processes

Putting a command into the background

Example: yum -y update & Output: [1] 21457

This only works when the user is logged on.

Bringing a command into the foreground

Use fg.

Example: fg %1 or fg 21457

Misc: kill %1 nukes it.

Audio/Video

Converting video file to sequence of still images

 ffmpeg -i video_file directory/filename.%04d.png

Piping (transferring) contents from one system to another via ssh

Instead of creating an archive on one system and then moving it, you have the option of piping the content through ssh and handling the contents how you choose on the other end. For these examples I'll use System1 and System2 to represent two different systems you want to transfer content between; rsync is another option.

Using tar/ssh to transfer current files from System1 to System2

# Logged into system2.example.org
cd /home
ssh myaccount@system1.example.org "cd /home && tar cf - myaccount" | tar xf -

Using tar/bzip2/ssh to backup current files from System1 and place on System2

# Logged into system2.example.org
cd /home
tar cf - my_home_dir | bzip2 -z --best |ssh myaccount@system1.example.org "cd /mnt/backups && cat > my_home_dir-yymmdd.tar.bz2"


Hostname configuration

Sad to say that I was probably improperly configuring system hostnames for years, but it turns out that it's pretty straightforward. Essentially, you configure the unqualified hostname in /etc/hostname and set the full qualified (FQDN) name in /etc/hosts.

cat /etc/hostname

node

cat /etc/hosts

127.0.0.1     localhost
192.168.1.10  node.example.org    node


References


curl

Using curl to check web server headers

Let's say for example that you want to make sure that you have the mime type configured properly for a Microsoft Word 2007 [1] or later document, and you have curl installed.

curl -I https://www.example.org/staff/docs/important.docx --user "USER_ACCOUNT"


You'll get back something that looks like:

Enter host password for user 'USER_ACCOUNT':
HTTP/1.1 200 OK
Date: Thu, 26 Jul 2012 13:10:20 GMT
Server: Apache/2.0.55 (Red Hat)
Last-Modified: Thu, 17 Nov 2011 14:12:00 GMT
ETag: "43d230-47cc-cdcdec00"
Accept-Ranges: bytes
Content-Length: 18380
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document

In this case, that is the Content-Type header that we want to provide so we're set.

References
  1. Content Types for Open XML Documents


Making a connection to an untrusted SSL site

Say for example that you need to access content on a SSL-enabled server that you trust, but you don't happen to have the CA certificate on that box (perhaps it is self-signed) and you want to connect anyway.

If the site requires a username/password, then this is how you would do it:

curl --user "USER_ACCOUNT" --insecure https://svn.example.org/example.org/systems/common/ubuntu/10.04.x/etc/apt/sources.list

If it doesn't, this is how you would connect:

curl --insecure https://svn.example.org/example.org/systems/common/ubuntu/10.04.x/etc/apt/sources.list


wget

Making a connection to an untrusted SSL site

Say for example that you need to access content on a SSL-enabled server that you trust, but you don't happen to have the CA certificate on that box (perhaps it is self-signed) and you want to connect anyway.

If the site requires a username/password, then this is how you would do it:

wget --user=USERNAME --ask-password --no-check-certificate  https://svn.example.org/example.org/systems/common/ubuntu/12.04.x/etc/apt/sources.list


If it doesn't, this is how you would connect:

wget --no-check-certificate https://svn.example.org/example.org/systems/common/ubuntu/12.04.x/etc/apt/sources.list

Updating a config file from SVN

wget --user=USERNAME --ask-password --no-check-certificate --output-document=/etc/apt/sources.list https://svn.example.org/example.org/systems/common/ubuntu/12.04.x/etc/apt/sources.list


Using rsync to mirror content

This synchronizes directory content between two systems, deleting any older content that is no longer present on the destination system.

rsync -arv --delete root@source.example.org:/home/user/ /home/user/

Spider your site to check for broken links

wget --wait=1 --waitretry=5 --spider -r -l inf --domains=example.org http://www.example.org/

Then take a look in your web server's error logs for what you need to fix.


Retrieve Serial Number/Model of an installed hard disk (without powering off system)

Useful in cases where you're having issues with a disk and need to replace it, but all of them are the same model & size.

If you have one or more SATA or SCSI hard disks installed and visible as /dev/sda, /dev/sdb and so forth, this is how you would query them for that information (serial numbers removed):

hdparm -I /dev/sd? | grep -E 'Number|/dev'
/dev/sda:
        Model Number:       WDC WD10EACS-00ZJB0
        Serial Number:      ...
/dev/sdb:
        Model Number:       WDC WD10EACS-00ZJB0
        Serial Number:      ...
/dev/sdc:
        Model Number:       WDC WD10EACS-00ZJB0
        Serial Number:      ...
/dev/sdd:
        Model Number:       WDC WD10EACS-00ZJB0
        Serial Number:      ...

If you were using IDE disks instead, just substitute /dev/hd? in place of /dev/sd?.


An alternative to hdparm is smartctl. I had some problems with hdparm not returning useful information:

hdparm -i /dev/sda

/dev/sda:
 HDIO_GET_IDENTITY failed: Invalid argument

whereas smartctl had no problems with the same disk:

smartctl -i /dev/sda

smartctl version 5.38 [i686-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model:     ST31000340NS
Serial Number:    XXXXXXX
Firmware Version: SN06
User Capacity:    1,000,204,886,016 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   8
ATA Standard is:  ATA-8-ACS revision 4
Local Time is:    Wed Nov 14 09:50:22 2012 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled


Here's a handy Bash one-liner that collects Model, Serial and Firmware info from each /dev/sdX disk:

for i in `ls /dev/sd?`; do echo -e "\n${i}" && smartctl -i $i | grep -E 'Serial|Model|Firmware'; done

References

Granting a group full access to content

For the purposes of this example, we're wanting to grant the group webmaster access to content within /var/www. This would be mainly to accommodate someone's personal preference as on Ubuntu systems the group www-data could be used for the same purpose.

Set permissions on directories

find /var/www -type d -exec chmod 775 {} \;

Set permissions on files

find /var/www -type f -exec chmod 664 {} \;

Set ownership on files/directories

chown -R www-data:webmaster /var/www

References

Distribution Specific

Enabling/Disabling Services starting at boot

Red Hat & CentOS

Enable
  1. Find out the name of the service's script from /etc/init.d/ directory. For example, mysqld or httpd [1]
  2. Enable it via chkconfig
    sudo /sbin/chkconfig --add mysqld
  3. Make sure it has been enabled
    sudo /sbin/chkconfig --list mysqld
  4. Set it to autostart
    sudo /sbin/chkconfig mysqld on
Disable
sudo /sbin/chkconfig mysqld off


Ubuntu

Enable

For example, let's say that we would like to set Apache v2.x to start automatically upon system boot.

  1. Get a list of services [2] via service --status-all
  2. We see that apache2 is the correct service name to reference and not apache or httpd.
  3. We set apache2 to auto-start [3] with
    update-rc.d apache2 defaults
Disable
  1. Get a list of services [2] via service --status-all
  2. We see that apache2 is the correct service name to reference and not apache or httpd.
  3. We disable apache2 from starting at boot [3] with
    update-rc.d apache2 disable

References

  1. abhigupta.com
  2. 2.0 2.1 Ubuntu Community Wiki - UbuntuBootupHowto page
  3. 3.0 3.1 Ubuntu(10.04 LTS) update-rc.d manpage


Using a local mirror

For example if you have a large number of servers that you're responsible for maintaining, it may be useful to mirror content locally and update all of your systems from that source instead of system outside of your network. One reason is a possible bandwidth savings, and another is a faster update source if you find yourself applying a lot of patches during maintenance hours. I found myself in that situation, and I'd often find remote update sources to be slow and unreliable.

CentOS

References


Ubuntu

References