Linux - Copy and follow symlinks

If you want to copy and follow symlinks, use these arguments:

cp -rfL source dest

Split a big text file into many files

Hi!

Suppose that we have a big text file with 2 000 000 lines and you want to split it in 1 000 files containing 20 000 lines each.

You only have one command to type!

split -l 20000 myfile.txt

You will get 1 000 files.

If you are on Windows, you just have to install Cygwin Terminal

Linux - find file in specific directory

If you want to search for all files under a specific directory and remove them, you could use the following command:

find . -type f -wholename '*/DIRECTORY/*' -exec rm -rf {} \;

Where DIRECTORY is the name of your directory.

For example, you will be able to find all theses files


/home/toto/test/DIRECTORY/toto.txt
/home/toto/test1/DIRECTORY/tata.txt
/home/toto/test2/DIRECTORY/titi.txt

You must execute the command from /home/toto

Batch Unzipping

Hi!

If you have a lot of files to unzip, you can do it by this single command.

 find . -name "*.gz" -exec gunzip {} \;

It will find all .gz files and unzip them.

IP Aliasing with ifconfig (Mac)

Hi

If you want to bind another IP address to your network card, you need to do IP aliasing. It will give you many IP address for a single nic.

It will help you if you need to start two server that listen on the same ports.

For example, I need to runs two differents systems that runs on 2 differents JBoss. I need to bind each JBoss to a specific IP to prevents port collision.

Here's how to add a new IP to your Mac or Linux.

Here's my IP informations

yannlaviolette@mac:~> ifconfig
lo0: flags=8049 mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
gif0: flags=8010 mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8963 mtu 1500
ether 00:1e:c2:0a:d6:2e
inet6 fe80::21e:c2ff:fe0a:d62e%en0 prefixlen 64 scopeid 0x4
inet 10.19.140.103 netmask 0xffffff00 broadcast 10.19.140.255
media: autoselect (100baseTX )
status: active
en1: flags=8863 mtu 1500
ether 00:1e:c2:a1:91:d1
inet6 fe80::21e:c2ff:fea1:91d1%en1 prefixlen 64 scopeid 0x5
inet 192.168.0.122 netmask 0xffffff00 broadcast 192.168.0.255
media: autoselect
status: active
fw0: flags=8863 mtu 4078
lladdr 00:1e:52:ff:fe:68:49:72
media: autoselect
status: inactive


To add a new IP, run this command: sudo ifconfig en0 alias 10.19.140.222 10.19.140.255 where 10.19.140.222 is the IP that I want to add and 10.19.140.255 is my broadcast address.

sudo ifconfig en0 alias 10.19.140.222 10.19.140.255

After the command I'm getting the following.

yannlaviolette@mac-jeanmat:~> ifconfig
lo0: flags=8049 mtu 16384
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
gif0: flags=8010 mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8963 mtu 1500
ether 00:1e:c2:0a:d6:2e
inet6 fe80::21e:c2ff:fe0a:d62e%en0 prefixlen 64 scopeid 0x4
inet 10.19.140.103 netmask 0xffffff00 broadcast 10.19.140.255
inet 10.19.140.222 netmask 0xff000000 broadcast 10.19.140.255
media: autoselect (100baseTX )
status: active
en1: flags=8863 mtu 1500
ether 00:1e:c2:a1:91:d1
inet6 fe80::21e:c2ff:fea1:91d1%en1 prefixlen 64 scopeid 0x5
inet 192.168.0.122 netmask 0xffffff00 broadcast 192.168.0.255
media: autoselect
status: active
fw0: flags=8863 mtu 4078
lladdr 00:1e:52:ff:fe:68:49:72
media: autoselect
status: inactive

To remove an IP, type the following command: sudo ifconfig en0 -alias 10.19.140.222 10.19.140.255

IMPORTANT!

If you need to ping your machine, you need to add a route to told your kernel that the new address is binded to 120.0.0.1. Type the following command

sudo route add -host 10.19.140.222 127.0.0.1

Detaching a shell with "screen"

If you want to protect yourself against connectivity lost with Unix / Linux, screen is the perfect command for you.

In a shell, type the command screen.

It will create a new shell.

Launch the command that you want and close your shell window.

We your are back, type screen -ls to show the list of active screen session.

You will get something like this:

[06:19:36]$ screen -ls
There are screens on:
    11944.pts-0.myserver    (Detached)
    4859.pts-3.myserver    (Detached)
    12190.pts-4.myserver    (Attached)
3 Sockets in /var/run/screen/S-user.


Attached => The shell window is still active.
Detached => No open shell window.



To join the screen session, type screen -r 11944.pts-0.myserver where the parameter is the session name.



To logout, simple type exit

Complex find exec zgrep example

Suppose that we have a filesystem containing EDI transactions. Here's an example for the filesystem content:

/home/myhome/myfolder/mysubfolder/toto/good/toto.gz
/home/myhome/myfolder/mysubfolder/toto/fail/toto.gz
/home/myhome/myfolder/mysubfolder/tata/good/tata.xml
/home/myhome/myfolder/mysubfolder/tata/fail/tata.gz
/home/myhome/myfolder/mysubfolder/foo/good/foo123.gz
/home/myhome/myfolder/mysubfolder/bar/fail/ef3d.gz

We need to copy in the current directory all the EDI transactions (text file starting with ISA) from each folders (toto, tata, foo, bar) but the folder must start with a T. In our example we need to copy files from "toto" and "tata". The good transactions are in the folder "good". We must not copy transactions that are not contained in other folder than "good".

Also, the EDI files may be compressed. So we need to verify in compressed files... "toto.gz" contains an EDI file and "tata.xml" is a xml file.

The command is:

find /home/myhome/myfolder/mysubfolder/ -type f -iwholename *good* -iwholename *mysubfolder\/t* -exec zgrep -l '^ISA' {}  \; -exec cp -rf {} . \;

Will copy only "toto.gz"


find => Searching for something
-type f => Looking for files
-iwholename => The path should contains a specific word
-exec zgrep -l '^ISA' {} => Look for file including compressed one. File must start with ISA
-exec cp -rf {} .  => Copy the data to the current directory

Remove a specific directory from hierarchy

To remove a directory named DIRECTORY_NAME from a directory hierarchy

find . -type d -iname DIRECTORY_NAME -exec rm -rf {} \;

Moving EDI files

Moving all EDI files where the name contains -- to a folder named EDI.

find . -type f -iname *--*  -exec grep -Rl '^ISA' {} \; -exec cp -rf {} EDI/ \;

GUnzip all found files

GUnzip all the files from a directory hierarchy

find . -type f -iname '*.gz' -exec gunzip {} \;

Want to have a verbose output?

find . -type f -iname '*.gz' -exec gunzip --verbose {} \;

Adobe Flash Player 11 on Fedora 16/15, CentOS/RHEL 6/5.7

Good tutorial that explains how to install flash on a CentOS / RedHat Linux machine

Adobe Flash Player 11 on Fedora 16/15, CentOS/RHEL 6/5.7

Virtualbox Guest Additions on CentOS & RedHat Linux

Check this URL if you want to configure the VirtualBox Guest additions on a Linux box.


http://www.if-not-true-then-false.com/2010/install-virtualbox-guest-additions-on-fedora-centos-red-hat-rhel/

EDI -=- Sample query to select and move transactions

Hi

If you have a bunch of EDI files, you may want to extract only specific transactions and move them to a folder. Here's a sample query. It moves all the PO (850) and the invoices (810) to the folder "MYFOLDER" in my home directory. The dot (.) in the grep represents any characters so this query will works will all delimiters.

find . -type f -exec grep -Rl 'ST.8[15]0' {} \; -exec cp -rf {} ~/MYFOLDER/ \;

Linux -=- Good reference for Regular Expression with GREP

Hi!

I've found a good reference that helps to understand regular expression with GREP.

Have a look at this site http://www.robelle.com/smugbook/regexpr.html

Yann

Linux -=- Gunzip all the files recursively

Hi!

If you have need to gunzip all the gzipped files that were under a directory hierarchy, you can type the following command. It will return all the files to delete and gunzip them. You must go in the root directory and type this command:

find . -iname '*.gz' -exec gunzip {} \;

Linux -=- Find files and copy to a folder

Hi!

Here's how to copy the files found in the result of a search to a specified folder.

find /path/to -iname 'whatever' -exec cp '{}' /path/to/destination/ \;

UNIX -=- GREP with OR operand

To find HR or FS in a file named myfile, type the following command in a terminal:

egrep 'HR|FS' myfile

Hex viewer in console

Hi!

If you want to view the hex content of a file under Mac OS X or Linux system, you can type in a console

hexdump -C file.xml

 It will show the hex content with the corresponding characters like this

00000000  3c 3f 78 6d 6c 20 76 65  72 73 69 6f 6e 3d 22 31  |00000010  2e 30 22 20 65 6e 63 6f  64 69 6e 67 3d 22 55 54  |.0" encoding="UT|

SSH Port Forwarding in 30 seconds

For those who want to access data behind a firewall, you can forward remote port onto a local port on your machine. For example, we want to forward the port 80 for abc.com to your localhost on port 8088. The only command you have to type is

ssh -L 8088:abc.com:80 user@abc.com

• 8088 is the local port on your computer
• abc.com is the distant host
• 80 is the distant port to forward
• user@abc.com is the credential used to log onto the host
  

SSH Login without password

In order to log from host A onto host B without a password, you need to add the RSA key from host A into the trusted key list of host B. In order to do that, follow theses steps:

1. Create RSA Key as a user. Don't enter any passphrase. The generated public key will be stored in HOME/.ssh/id_rsa.pub

   ssh-keygen -t rsa

2. Create on host B .ssh folder

   mkdir ~/.ssh

3. Export RSA public key to host B with this command on host A

   cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'

4. Be sure that the permission are 700. You can this command to set permission

   chmod -R 700 ~/.ssh