Search This Blog

Tuesday, 27 December 2016

Unix Shell Scripting Commands - Part 1

Unix Shell Scripting :

We interact with the Unix kernel with the help shell application (which is called Terminal)
All unix commands have to be in lower case.

clear : clears the screen

cd ~ : Navigate to root folder

who am I : displays  three colums ( username, terminal id, time and date of login)

pwd  : shows the directory where the shell is in.  (pwd stands for present working directory)

cal : shows the calendar of the current month and year and current date is highlighted

cal 8 2007 or cal aug 2007 : calendar of august 2007 will be displayed

date : displays the date ( displays in default format)

To display date in custom format , then we do in the following way
date  ‘+DATE: %m-%y%Ntime:%H%M%S’ :
if you observe in the above command we have written in capital letters, that is not a command,that is to define format , so it can be written in capital letters

touch file1 file2  : this commands creates files in home directory

mkdir testdir :  creates a directory with name testdir in the home directory

mkdir documents/testdir : creates a directory in the documents path

cd testdir : changes the present working directory to the testdir , to create a file and content

cat > test : write the contents of the file, ctrl + D to end the editing

cat < test : to show the contents of the file.

Merge two files  cat > file1 file2 > file3  : contents of file1 and file2 will be merged and copied into file3

Rename and Delete :
mv oldfilename newfilename  : this renames the file or dire to new name given
rm filename : file will be deleted
rm –r directoryname  :  to deleted the directory
rmdir directoryname : also deleted the directory

Copies :
cp file_to_be_copied  destinationpath  : copies the file to the destination path
Links
   Hardlinks : creates a replica : ln old new :
   Softlinks : create a shortcut

File Permissions :
Every file or directory have below types of permissions
Owner
Read/Write
Group having access
Others

umask :  displays the four digit no , ignore the first digit , and substract the number from 666 if it is a file and from 777 if it is a directory,
ex : if the value 0022 for a file, subtract the value from 666 which give 644, that means the owner has read and write permissions where as the group and others have just read access.

chmod :  is used to modify file permissions of a file or directory
   ex : chmod 666 filename   : gives read and write to all for a file

Contents of directory:
      ls : lists the folders and files in a file of the home directory
       ls directoryname : 
       ls - l : longlist .
       ls – a : lists all the files along with hidden files
        file * : displays the contents of the folder, says the type of file ,directory etc
        ls –ltra  (shows all the hidden files
       ls –F : helps us to differentiate between files and directories

   
OS version details
uname –a : to now the version of the os and 

Count characters, words and lines:
1.wc filenametobeevaluated : displays the no of lines, words and characters
2.wc –l filenametobeevaluated :If we want only no of line then simply use
    

Sorting :
sort filetobesorted : the lines in the file will be sorted





Spark Memory Management

 Spark’s performance advantage over MapReduce  is due to Spark’s In-Memory Persistence and Memory Management Rather than writing to disk ...