Search all files containing a specific string

How to search all files containing a specific string on Linux and Windows?

  • On Linux

    grep -rnw ‘directory’ -e “pattern”
    grep –include=\*.{txt,log} -rnw ‘directory’ -e “pattern”
    This will only search for files with .txt or .log extension.
    grep –exclude=*.txt -rnw ‘directory’ -e “pattern”
    This will exclude files with .txt extensions.

  • On Windows

    CD Location
    FINDSTR /L /S /I /N /C:”pattern” *.log

Browse the internet via command line

How to browse the internet via command line on Linux and Windows?

  • On Linux

    lynx http://google.ca
    If you don’t have lynx on your linux installation, you will have to install it. On Linux Red hat, install it like this:
    yum list lynx (to check the availability of the package)
    yum -y install lynx (to install the package)
    you can also use: curl -0 http://yoursite/index.html to get the source code of a specific file.

  • On Windows

    start /max http://google.ca
    Will open the url using your default browser.

un-archive a folder silently in Windows

Powershell script to un-archive a folder silently in Windows

Let’s continue with our automation scripting tool series. Doing things on windows by using the mouse manually is fun until you run into the need to go to a major OS upgrade and have tons of applications to reinstall. To avoid human errors and save lots of time, you can write a script  to pick up your archived applications like tomcat zip files and unarchive it. This powershell scripts shows you how to unarchive a zip file silently.

#Unzip FIle Function
function Expand-ZIPFile($file, $destination)
{
$shell = new-object -com shell.application
$zip = $shell.NameSpace($file)
foreach($item in $zip.items())
{
$shell.Namespace($destination).copyhere($item)
}
}

# Call the function to unzip

$Origin_Zip_File=”C:\Document\Test.zip”

$Destination_Folder=”C:\Document\Test_Unzipped”

Expand-ZIPFile –File $Origin_Zip_File –Destination $Destination_Folder

Script to Stop Application Pools on Windows

VB script to Stop Application Pools on Windows IIS 6

As  SysAdmin, We sometimes need to auto-stop some application pools in order to run a specific task like IIS logs cleanup.

This script will detect all application pools that are running and stop them.

=====================
myComputer = “.”
Set objWMIService = GetObject (“winmgmts:\\” & myComputer & “\root\microsoftiisv2”)
Set colItems = objWMIService.ExecQuery (“Select name from IISApplicationPoolSetting where AppPoolState = 2”)
For Each objItems In colItems
poolname=objItems.name
WScript.Echo poolname
AppPoolStop(poolname)
Next
Function AppPoolStop(PoolName)
set PoolcolItems = objWMIService.ExecQuery (“Select name from IISApplicationPool where name='” & poolname & “‘”)
For Each objItem In PoolcolItems
objitem.Stop
Next
End Function
===============
Enter that script in a text file, save it with extension “.vbs”. Copy it at C:\Windows\System32\
Run it manually or via a batch scheduler.
Thanks to Euk-marie. See the equivalent script by Euk-Marie to start the App Pool at https://www.eukhost.com/forums/f15/start-application-pool-simple-script-8937/

kill all processes used by a specific application or user

Shell script to kill all processes used by a specific application or user

On Linux servers, when applications are not correctly shut down, there are still some processes running and starting the application with those processes still running can result in the application not starting correctly or not functioning properly. It is imperative to always kill all lingering processes after shutting down an application.

If the list of  the lingering processes is long, it can be a pain to go through them one by one.

This shell script will kill them all in one command.

#!/bin/sh
#NOTE: Please don’t try this unless you have tried every thing else and there is no way else left.
# Run it as root
# For this example, you are running a oracle application using user account oracle.
# We assume that , there are not other application running under the account oracle.
kill -9 `ps -ef | grep oracle  | grep -v grep | awk ‘{print $2}’`

Linux & App Servers Monitoring Tricks

LINUX & APPLICATION SERVERS MONITORING TRICKS


Objective:

  • Prevent server from going down,
  • detect what caused the server to go down,
  • get servers back after failure.
  • Explain what caused the server to fail

I – Regularly watch your monitoring  tool(s) (Nagios, Wily,top, …..)

On some of those tools,  you can see a graph of the apps CPU Load Average, CPU Used Percentage, Disk Usage percentage, memory used percentage, Network Bandwidth, Swap Used percentage.

II – Check ulimit count (Number of files opened by applications like tomcat, oracle,…)

[Server]# ulimit -n
1024
If the number of files opened is getting closer to ulimit count (1024), increase the ulimit and talk to dev to identify and fix the process that is causing that.
To increase ulimit count for a specific application account, run the command  ulimit –n [value]
Ulimit can be set to whatever you want. Its one of those things that’s put in place as a throttle to keep things from going too nuts. Some systems will actually just set it to unlimited.

III- Port Monitoring 
Check number of connections to ports used by your apps

IV- Thread dump (stack trace of all threads ) If you have a high cpu percentage

[Server]# kill -3 (The output is printed in catalina.out) to see what is causing this and send it to developers.

V- Disk space /[drive_name] filling up quickly
Identify the file(s) that are filling up the disks. Most of the time ,it will be logs files.
[Server]# du -ks /[drive_name]/* | sort -nr | head
5719076 /[drive_name]/catalina
3675672 /[drive_name]/data
3287436 /[drive_name]/source
2044316 /[drive_name]/servers
319404 /[drive_name]/images
16 /[drive_name]/lost+found
By running this command on the larger folder, that will lead you to the files that eat the disk space.
Back up, remove or empty the file in question given that it won’t break the system.
If the log files are responsible for the disk filling up, let the developer know about it so that they can solve it. In the meantime, empty the log file with the command:

[Log_File_Location]# echo -n > Large_Log_File_Name.log

VI- Watch catalina.out and log4j.out after staging and live deploy, especially when you are restarting the servers.

 

[Server]# tail -f log4j.log
VII- Start app servers properly
Before restarting app servers, make sure there is no app pid running for that specific server.

[Server_Name]$ ps -ef | grep oracle
Kill the pid for that server.

IX – Cpu Load level

I would say that if we peak under 70% CPU during high traffic, we are doing well and have room. A good level to be ticking over at would be 30% used.
[Server]# top
top – 12:37:29 up 47 days, 23:09, 4 users, load average: 0.20, 0.20, 0.22
Tasks: 189 total, 1 running, 178 sleeping, 10 stopped, 0 zombie
Cpu(s): 1.2%us, 0.1%sy, 0.0%ni, 97.5%id, 1.0%wa, 0.0%hi, 0.1%si, 0.0%st

X- Server specific status pings (To assure the server are up and serving contents)
Write scripts for this

XI- Garbage collection stats

If you are interested in any garbage collection stats there’s the gc.log files on each of the appservers (bad thing about it is it doesn’t do any date stamping so you can see how memory fluctuates but its a difficult to create a chart over time). In the past I’ve thought it might be good idea to write a cron that archived it daily so that you could at least break things down day by day.

XII- DB Connection

XIII- Load Average Monitoring script
Set up a  cron that just email sysadmin when the load average is above 3.

XIV – Find out who is monopolizing or eating the CPUs
[Server]# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10