This is how to How to call a shell script from python code
import subprocess
subprocess.call(['sh shell_script.sh arguments'])
or
import sys, os
os.system("sh shell_script.sh arguments")
Output Example
Basic Gotcha Linux Questions for IT DevOps and SysAdmin Interviews
Some IT DevOps, SysAdmin, Developer positions require the knowledge of basic linux Operating System. Most of the time, we know the answer but forget them when we don’t practice very often. This refresher will help you prepare for the linux portion of your IT interview by answering some gotcha Linux Questions for IT DevOps and SysAdmin Interviews.
Latest Linux Feeds
I- Networking:
- How many bytes are there in a MAC address?
48.
MAC, Media Access Control, address is a globally unique identifier assigned to network devices, and therefore it is often referred to as hardware or physical address. MAC addresses are 6-byte (48-bits) in length, and are written in MM:MM:MM:SS:SS:SS format. - What are the different parts of a TCP packet?
The term TCP packet appears in both informal and formal usage, whereas in more precise terminology segment refers to the TCP protocol data unit (PDU), datagram to the IP PDU, and frame to the data link layer PDU: … A TCP segment consists of a segment header and a data section. - Networking: Which command is used to initialize an interface, assign IP address, etc.
ifconfig (interface configuration). The equivalent command for Dos is ipconfig.
Other useful networking commands are: Ping, traceroute, netstat, dig, nslookup, route, lsof - What’s the difference between TCP and UDP; Between DNS TCP and UDP?
There are two types of Internet Protocol (IP) traffic. They are TCP or Transmission Control Protocol and UDP or User Datagram Protocol. TCP is connection oriented – once a connection is established, data can be sent bidirectional. UDP is a simpler, connectionless Internet protocol.
The reality is that DNS queries can also use TCP port 53 if UDP port 53 is not accepted.
DNS uses TCP for Zone Transfer over port :53.
DNS uses UDP for DNS Queries over port :53. - What are defaults ports used by http, telnet, ftp, smtp, dns, , snmp, squid?
All those services are part of the Application level of the TCP/IP protocol.
http => 80
telnet => 23
ftp => 20 (data transfer), 21 (Connection established)
smtp => 25
dns => 53
snmp => 161
dhcp => 67 (server), 68 (Client)
ssh => 22
squid => 3128 - How many host available in a subnet (Class B and C Networks)
- How DNS works?
When you enter a URL into your Web browser, your DNS server uses its resources to resolve the name into the IP address for the appropriate Web server. - What is the difference between class A, class B and class C IP addresses?
Class A Network (/ 8 Prefixes)
This network is 8-bit network prefix. IP address range from 0.0.0.0 to 127.255.255.255
Class B Networks (/16 Prefixes)
This network is 16-bit network prefix. IP address range from 128.0.0.0 to 191.255.255.255Class C Networks (/24 Prefixes)
This network is 24-bit network prefix.IP address range from 192.0.0.0 to 223.255.255.255 - Difference between ospf and bgp?
The first reason is that BGP is more scalable than OSPF. , and this, normal igp like ospf cannot perform. Generally speaking OSPF and BGP are routing protocols for two different things. OSPF is an IGP (Interior Gateway Protocol) and is used internally within a companies network to provide routing.
II- Operating System
1&1 Web Hosting
- How to find the Operating System version?
$uname -a
To check the distribution for redhat for example: $cat /etc/redhat –release - How to list all the process running?
top
To list java processes, ps -ef | grep java
To list processes on a specific port:
netstat -aon | findstr :port_number
lsof -i:80 - How to check disk space?
df shows the amount of disk space used and available.
du displays the amount of disk used by the specified files and for each subdirectories.
To drill down and find out which file is filling up a drive: du -ks /drive_name/* | sort -nr | head - How to check memory usage?
free or cat /proc/meminfo - What is the load average?
It is the average sum of the number of process waiting in the queue and the number of process currently executing over the period of 1, 5 and 15 minutes. Use top to find the load average. - What is a load balancer?
A load balancer is a device that acts as a reverse proxy and distributes network or application traffic across a number of servers. Load balancers are used to increase capacity (concurrent users) and reliability of applications. - What is the Linux Kernel?
The Linux Kernel is a low-level systems software whose main role is to manage hardware resources for the user. It is also used to provide an interface for user-level interaction. - What is the default kill signal?
There are many different signals that can be sent (see signal for a full list), although the signals in which users are generally most interested are SIGTERM (“terminate”) and SIGKILL (“kill”). The default signal sent is SIGTERM.
kill 1234
kill -s TERM 1234
kill -TERM 1234
kill -15 1234 - Describe Linux boot process
BIOS => MBR => GRUB => KERNEL => INIT => RUN LEVEL
As power comes up, the BIOS (Basic Input/Output System) is given control and executes MBR (Master Boot Record). The MBR executes GRUB (Grand Unified Boot Loader). GRUB executes Kernel. Kernel executes /sbin/init. Init executes run level programs. Run level programs are executed from /etc/rc.d/rc*.d
Mac OS X Boot Process:Boot ROM Firmware. Part of Hardware system
BootROM firmware is activatedPOST Power-On Self Test
initializes some hardware interfaces and verifies that sufficient memory is available and in a good state.EFI Extensible Firmware Interface
EFI does basic hardware initialization and selects which operating system to use.BOOTX boot.efi boot loader
load the kernel environmentRooting/Kernel The init routine of the kernel is executed
boot loader starts the kernel’s initialization procedure
Various Mach/BSD data structures are initialized by the kernel.
The I/O Kit is initialized.
The kernel starts /sbin/mach_initRun Level mach_init starts /sbin/init
init determines the runlevel, and runs /etc/rc.boot, which sets up the machine enough to run single-user.
rc.boot figures out the type of boot (Multi-User, Safe, CD-ROM, Network etc.) - List services enabled at a particular run level
chkconfig –list | grep 5:0n
Enable|Disable a service at a specific run level: chkconfig on|off –level 5 - How do you stop a bash fork bomb?
Create a fork bomb by editing limits.conf:
root hard nproc 512
Drop a fork bomb as below:
:(){ :|:& };:
Assuming you have access to shell:
kill -STOP
killall -STOP -u user1
killall -KILL -u user1 - What is a fork?
fork is an operation whereby a process creates a copy of itself. It is usually a system call, implemented in the kernel. Fork is the primary (and historically, only) method of process creation on Unix-like operating systems. - What is the D state?
D state code means that process is in uninterruptible sleep, and that may mean different things but it is usually I/O.
III- File System
- What is umask?
umask is “User File Creation Mask”, which determines the settings of a mask that controls which file permissions are set for files and directories when they are created. - What is the role of the swap space?
A swap space is a certain amount of space used by Linux to temporarily hold some programs that are running concurrently. This happens when RAM does not have enough memory to hold all programs that are executing.
- What is the role of the swap space?
A swap space is a certain amount of space used by Linux to temporarily hold some programs that are running concurrently. This happens when RAM does not have enough memory to hold all programs that are executing. - What is the null device in Linux?
The null device is typically used for disposing of unwanted output streams of a process, or as a convenient empty file for input streams. This is usually done by redirection. The /dev/null device is a special file, not a directory, so one cannot move a whole file or directory into it with the Unix mv command.You might receive the “Bad file descriptor” error message if /dev/null has been deleted or overwritten. You can infer this cause when file system is reported as read-only at the time of booting through error messages, such as“/dev/null: Read-only filesystem” and “dup2: bad file descriptor”.
In Unix and related computer operating systems, a file descriptor (FD, less frequently fildes) is an abstract indicator (handle) used to access a file or other input/output resource, such as a pipe or network socket. - What is a inode?
The inode is a data structure in a Unix-style file system that describes a filesystem object such as a file or a directory. Each inode stores the attributes and disk block location(s) of the object’s data.
IV- Databases
- What is the difference between a document store and a relational database?
In a relational database system you must define a schema before adding records to a database. The schema is the structure described in a formal language supported by the database and provides a blueprint for the tables in a database and the relationships between tables of data. Within a table, you need to define constraints in terms of rows and named columns as well as the type of data that can be stored in each column.In contrast, a document-oriented database contains documents, which are records that describe the data in the document, as well as the actual data. Documents can be as complex as you choose; you can use nested data to provide additional sub-categories of information about your object. You can also use one or more document to represent a real-world object. - How to optimise a slow DB?
- Rewrite the queries
- Change indexing strategy
- Change schema
- Use an external cache
- Server tuning and beyond
- How would you build a 1 Petabyte storage with commodity hardware?
Using JBODs with large capacity disks with Linux in a distributed storage system stacking nodes until 1PB is reached.
JBOD (which stands for “just a bunch of disks”) generally refers to a collection of hard disks that have not been configured to act as a redundant array of independent disks (RAID) array.
V- Scripting
- What is @INC in Perl?
The @INC Array. @INC is a special Perl variable that is the equivalent to the shell’s PATH variable. Whereas PATH contains a list of directories to search for executables, @INC contains a list of directories from which Perl modules and libraries can be loaded. - Strings comparison – operator – for loop – if statement
- Sort access log file by http Response Codes
Via Shell using linux commands
cat sample_log.log | cut -d ‘”‘ -f3 | cut -d ‘ ‘ -f2 | sort | uniq -c | sort -rn - Sort access log file by http Response Codes Using awk
awk ‘{print $9}’ sample_log.log | sort | uniq -c | sort -rn - Find broken links from access log file
awk ‘($9 ~ /404/)’ sample_log.log | awk ‘{print $7}’ sample_log.log | sort | uniq -c | sort -rn - Most requested page:
awk -F\” ‘{print $2}’ sample_log.log | awk ‘{print $2}’ | sort | uniq -c | sort -r - Count all occurrences of a word in a file
grep -o “user” sample_log.log | wc -w
Learn more at http://career.guru99.com/top-50-linux-interview-questions/
Real Time Linux Jobs
Script with hash tables on windows and Linux
How to declare and write a script with hash tables on windows and linux
-
Hash tables with powershell on windows
Declaration:
$states=@{“Alberta” = “Calgary”; “British Columbia” = “Vancouver”; “Ontario” = “Toronto” ; “Quebec” = “Montreal”}Name
_____Value
_______Alberta Calgary British Columbia Vancouver Ontario Toronto Quebec Montreal Add new key-value in hashtable:
$states.Add(“Manitoba”,”Winnipeg”)Remove key-value in hashtable:
$states.Remove(“Manitoba”,”Winnipeg”)
Change value in hashtable:
$states.Set_Item(“Ontario”,”Ottawa”)
Retrieve value in hashtable:
$states.Get_Item(“Alberta”)
Find key in hashtable:
$states.ContainsKey(“Alberta”)
Find Value in hashtable:
$states.ContainsValue(“Calgary”)
Count items in hashtable:
$states.Count
Sort items by Name in hashtable:
$states.GetEnumerator() | Sort-Object Name -descending
Sort items by Value in hashtable:
$states.GetEnumerator() | Sort-Object Value -descending -
Hash tables with perl on linux or windows
Declaration:
my %hash = (); #Initialize a hash
my $hash_ref = {}; # Initialize a hash reference. ref will return HASH
Clear (or empty) a hash
for (keys %hash)
{
delete $hash{$_};
}
Clear (or empty) a hash reference
for (keys %$href)
{
delete $href->{$_};
}
Add a key/value pair to a hash
$hash{ ‘key’ } = ‘value’; # hash
$hash{ $key } = $value; # hash, using variables
Using Hash Reference
$href->{ ‘key’ } = ‘value’; # hash ref
$href->{ $key } = $value; # hash ref, using variables
Add several key/value pairs to a hash
%hash = ( ‘key1’, ‘value1’, ‘key2’, ‘value2’, ‘key3’, ‘value3’ );
%hash = (
key1 => ‘value1’,
key2 => ‘value2’,
key3 => ‘value3’,
);Copy a hash
my %hash_copy = %hash; # copy a hash
my $href_copy = $href; # copy a hash ref
Delete a single key/value pair
delete $hash{$key};
delete $hash_ref->{$key};
Hash tables with python on linux or windows
Hash tables are called dictionary in python.
Declaration:
dict = {‘Name’: ‘Zara’, ‘Age’: 7, ‘Class’: ‘First’}
Accessing Values
print “dict[‘Name’]: “, dict[‘Name’]
print “dict[‘Age’]: “, dict[‘Age’]
Output:
dict[‘Name’]: Zara
dict[‘Age’]: 7
Updating Dictionary
dict = {‘Name’: ‘Zara’, ‘Age’: 7, ‘Class’: ‘First’}
dict[‘Age’] = 8; # update existing entry
dict[‘School’] = “DPS School”; # Add new entry
Delete Dictionary Elements
#!/usr/bin/python
dict = {‘Name’: ‘Zara’, ‘Age’: 7, ‘Class’: ‘First’}
del dict[‘Name’]; # remove entry with key ‘Name’
dict.clear(); # remove all entries in dict
del dict ; # delete entire dictionary
Source:
- https://technet.microsoft.com/en-us/library/ee692803.aspx
- http://www.cs.mcgill.ca/~abatko/computers/programming/perl/howto/hash/
- http://www.tutorialspoint.com/python/python_dictionary.htm
reverse a string on Linux and Windows
How to reverse a string on Linux and Windows
Via shell script on Linux
sh-3.2# vi reverse.sh
#### Start Script #####
#!/bin/bash
input_string=”$1″
reverse_string=””
input_string_length=${#input_string}
for (( i=$input_string_length-1; i>=0; i– ))
do
reverse_string=”$reverse_string${input_string:$i:1}”
done
echo “$reverse_string”
##### End Script #####
Let’s run it:
sh-3.2# chmod 775 reverse.sh
sh-3.2# ./reverse.sh Etienne
enneitE
Via powershell script on Windows
#Let’s use the script reverse.ps1 below.
######
$string=”Etienne”
$string_array=$string -split “”
[array]::Reverse($string_array)
$string_array -join ”
#####Output#####
PS C:\Users\etienne_noumen\Documents\Etienne\Scripting> .\reverse.ps1
E t i e n n e
enneitE
Via powershell script on Windows in one line
([regex]::Matches($String,’.’,’RightToLeft’) | ForEach {$_.value}) -join ”
Via batch script on Windows
::Note: ReverseStr also calls StrLen
::and string length is not greater than 80 chars
:: but can be changed.
@echo off
SetLocal EnableDelayedExpansion
cls
set Str=Etienne
call :StrLen %Str%
echo Length=%Len%
call :ReverseStr %Str%
echo String=%Str%
echo Reverse Str=%Reverse%
exit /b
::—————-
:: Calc Var Length
::—————-
:: %*=Str to Check
:: Returns %Len%
:: —————
:StrLen %*
set Data=%*
for /L %%a in (0,1,80) do (
set Char=!Data:~%%a,1!
if not “!Char!”==”” (
set /a Len=%%a+1
) else (exit /b)
)
exit /b
::—————
:: Reverse String
::—————
:: %* Str to Reverse
:: Returns %Reverse%
::——————
:ReverseStr %*
set Data=%*
call :StrLen %Data%
for /L %%a in (!Len!,-1,0) do (
set Char=!Data:~%%a,1!
set Reverse=!Reverse!!Char!
)
exit /b
Via perl script on Windows or Linux
Via python script on Windows or Linux
def reverse_string(a_string)
return a_string[::-1]
reverse_string(“etienne”) returns “enneite”
Source:
Linux Boot process
Let’s describe Linux Boot Process:
| BIOS |
Basic INPUT/OUTPUT System. Executes MBR |
| MBR |
Master Boot Record Executes GRUB |
| GRUB |
Grand Unified Bootloader Executes kernel |
| KERNEL |
Kernel Executes /sbin/init |
| INIT |
Init Executes Run level programs |
| Run Level | Run Level Programs are executed from /etc/rc.d/rc*.d/ |
- As power comes up the BIOS is given control
- BIOS runs self tests, usually including cursory memory tests.
- The BIOS then loads the first sector of the disk to be used for booting and transfers control to it.
- The MBR code varies. One version will chain to the code in the first sector of the boot partition (Windows), another will load a bootloader. Windows boot proceeds from code and information in the boot partition.
- The bootloader chooses kernel location and version
- The bootloader prepares kernel and initrd image in memory, transfers control to kernel
- Loading kernel modules
- Discovering hardware and load additional kernel modules to support it
- Looking for disks
- R/O mount of / partition so that it can potentially be checked and repaired
- init process spawn
- /etc/inittab read and executing
- Mounting all FSes from /etc/fstab
- runlevels running (based on default runlevel in /etc/inittab) or another init method such as systemd or upstart
- rc.local
- login prompt
Source:
- http://unix.stackexchange.com/questions/27034/discribe-in-details-tha-boot-process-of-any-linux-system
- http://www.thegeekstuff.com/2011/02/linux-boot-process/
- Watch video here
- http://www.slideshare.net/dominiquec/understanding-the-boot-process
Monitor Macbook
How to Monitor Macbook with one single command?
$sudo sysdiagnose -f ~/Desktop/
The result is a compressed file named sysdiagnose_YYYY.MM.DD_HH-MM-SS-TTTT.tar.gz and it contains the following:
Accessibility
BluetoothTraceFile.pklg
DiagnosticMessages
Etienne’s SystemConfiguration
airport_info.txt
apsd-status.txt
bc_stats.txt
bootstamps.txt
brctl.tar.gz
breadcrumbs.txt
crashes_and_spins
darwinup.txt
dig-results.txt
disks.txt
diskutil.txt
error_log.txt
filecoordination_dump.txt
footprint-all.txt
fs_usage.txt
fsck_hfs_user.log
fsck_hfs_var.log
gpt.txt
ifconfig.txt
ioreg
ipconfig.txt
kextstat.txt
launchctl-list.txt
locale.txt
logs
lsappinfo.txt
lsmp.txt
lsof.txt
lsregister.txt
microstackshots
microstackshots_lastday.txt
microstackshots_lasthour.txt
microstackshots_lastminute.txt
mount.txt
netstat
nfsstat.txt
odutil.txt
pluginkit.txt
pmset_everything.txt
powermetrics.txt
ps.txt
ps_thread.txt
reachability-info.txt
resolv.conf
scutil.txt
smcDiagnose.txt
spindump.txt
stackshot-last-sym.log
sysctl.txt
sysdiagnose.log
system_profiler.spx
talagent.txt
taskinfo.txt
thermal.txt
top.txt
var_run_resolv.conf
vm_stat.txt
zprint.txt
List only regular file names in a directory
How to List only regular file names in a directory on Linux and Windows
Listing regular files in a directory without including . and .. files.
On Linux
Solution 1:$ ls -p | grep -v /
Solution 2: $ ls -F | grep -v ‘[/@=|]’
Solution 3: $for list in `ls` ; do ls -ld $list | grep -v ^d > /dev/null && echo $list ; done ;
Solution4:$ for list in `ls` ; do ls -ld $list | grep ^d > /dev/null || echo $list ; done ;
Solution5 (exclude sym links):$ for list in `ls` ; do ls -ld $list | grep -v ^l > /dev/null && echo $list ; done ;
On Windows
Solution 1: dir /a-d /b >..\File_List.txt
Set Date and time via command line
Let’s find out how to set Date and time via command line on linux and windows:
-
On Linux via terminal
System time (Must have sudo privilege)
date -s ‘2015-07-28 15:27:30’
Hardware time
Let’s set the hardware clock to the current system time:
hwclock –systohc
-
On Windows via command prompt terminal
System time (Must have Administrator privilege)
date
The current date is: 07/28/2015
Enter the new date: (mm-dd-yy)_
time
The current time is: 15:34:03.44
Enter the new time: _
prompt and read input variables from keyboard
Let’s find how to prompt and read input variables from keyboard while executing a script using shell, perl, python, batch and powershell (windows and Linux)
-
On Linux via shell
read -p “Enter your name: ” name
echo “Hi, $name. Let’s be friend!”
-
On Windows via powershell
$name=read-host “Enter your name:”
write-host “Hi $name, Let’s be friend!” -
On Windows via batch
Set /p Name=”Enter your name:”
echo “Hi %name%, Let’s be friend!” -
On Windows or Linux via perl
print “Enter your name “;
my $name =;
chomp $name; # Get rid of newline character at the end
print “Hello $name, let’s be friend”; -
On Windows or Linux via python
name=input(“Enter your name: “)
print (“Hello ” + name + ” let’s be friend”)
List all processes holding a specific port
How to list all processes holding a specific port on Linux and windows via command line
Let’s find all processes pid using port 80 on linux and windows:
-
On Linux
lsof -i:80
-
On Windows
netstat -aon | findstr :80