The Proc filesystem and getting information on a process,cpu,memory

Getting PID of a process 

If we know the name of the application or script running , then we can get it's process id using a ps command.

ps -ef | grep <application/script name >

eg )

ps command

 In my case , i got the process id of google-chrome browser as 2233. Chrome has a multi process architecture ,hence multiple pids in the out put.

PSTREE .

I can get the process hierarchy of a process using the pstree command .

eg)

If we run the pstree command without any pid as parameter ,we will get a process map of all the services and process running in the machine .

eg)pstree

/proc Filesystem 

In linux ,everything is a file including hardware devices and running processes .The /proc filesystem has all the information about features and  usage of cpu , memory , details of all the applications running in the machine .

For instance , let us take the chrome process running in my machine .

There will be a directory in /proc filesystem for each process , it will have files to store information (different properties ) about the process. The path will be like 

/proc/<pid as directory name>/<files>

eg) for chrome 

[prabu@dev sysctl.d]$ more /proc/2233/

attr/            comm             fd/              maps             ns/              personality      stack            uid_map

autogroup        coredump_filter  fdinfo/          mem              numa_maps        projid_map       stat             wchan

auxv             cpuset           gid_map          mountinfo        oom_adj          root/            statm            

cgroup           cwd/             io               mounts           oom_score        sched            status           

clear_refs       environ          limits           mountstats       oom_score_adj    sessionid        syscall          

cmdline          exe              loginuid         net/             pagemap          smaps            task/            

ie)

2233 is the directory in process filesystem and the files and sub directories are below 

The name of program of the process is stored in comm file 

eg)

[prabu@dev sysctl.d]$ more /proc/2233/comm

chrome

The full command of the process along with path is in cmdline

eg)

[prabu@dev sysctl.d]$ more /proc/2233/cmdline 

/opt/google/chrome/chrome 

The userid in which it runs is in loginuid

eg)

[prabu@dev sysctl.d]$ more /proc/2233/loginuid 

1000

[prabu@dev sysctl.d]$ id

uid=1000(prabu) gid=1000(prabu) groups=1000(prabu),10(wheel)

The io details like amount of data processed by the application is in io file.

eg)

[prabu@dev sysctl.d]$ more /proc/2233/io

rchar: 161425371

wchar: 111930271

syscr: 1465561

syscw: 760250

read_bytes: 327160320

write_bytes: 79355904

cancelled_write_bytes: 434176

The limits on resources that can be used by the process is in limits file .

eg)

[prabu@dev sysctl.d]$ more /proc/2233/limits 

Limit                     Soft Limit           Hard Limit           Units     

Max cpu time              unlimited            unlimited            seconds   

Max file size             unlimited            unlimited            bytes     

Max data size             unlimited            unlimited            bytes     

Max stack size            8388608              unlimited            bytes     

Max core file size        0                    unlimited            bytes     

Max resident set          unlimited            unlimited            bytes     

Max processes             4096                 14784                processes 

Max open files            1024                 4096                 files     

Max locked memory         65536                65536                bytes     

Max address space         unlimited            unlimited            bytes     

Max file locks            unlimited            unlimited            locks     

Max pending signals       14784                14784                signals   

Max msgqueue size         819200               819200               bytes     

Max nice priority         0                    0                    

Max realtime priority     0                    0                    

Max realtime timeout      unlimited            unlimited            us        

The status information of process is in status ,

eg)

[prabu@dev sysctl.d]$ more /proc/2233/status

Name: chrome

State: S (sleeping)

Tgid: 2233

Ngid: 0

Pid: 2233

PPid: 1940

TracerPid: 0

Uid: 1000 1000 1000 1000

Gid: 1000 1000 1000 1000

FDSize: 512

Groups: 10 1000 

VmPeak: 1773700 kB

VmSize: 1733864 kB

Similarly there are sub directories for file info in fd (fiels used by process ) and mem file which has info on memory segments allocated ,pagemap file which has information on pages related to the process.

Getting CPU details of the machine .

  The details on architecture and features of cpu ,no of cores and threads can be obtained using below command. 

 cat /proc/cpuinfo :

eg)

[prabu@dev sysctl.d]$ cat /proc/cpuinfo |more

processor : 0

vendor_id : GenuineIntel

cpu family : 6

model : 37

model name : Intel(R) Pentium(R) CPU        P6200  @ 2.13GHz

stepping : 5

microcode : 0x4

cpu MHz : 2133.000

cache size : 3072 KB

physical id : 0

siblings : 2

core id : 0

cpu cores : 2

apicid : 0

initial apicid : 0

fpu : yes

fpu_exception : yes

cpuid level : 11

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx r

dtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm pcid

 popcnt lahf_lm arat dtherm

bogomips : 4256.20

clflush size : 64

cache_alignment : 64

address sizes : 36 bits physical, 48 bits virtual

power management:

processor : 1

vendor_id : GenuineIntel

cpu family : 6

model : 37

model name : Intel(R) Pentium(R) CPU        P6200  @ 2.13GHz

stepping : 5

.

.

Getting details of memory available, used and pagesize .

Details on total available memory ,used memory ,type ,size and no of pages can be obtained by below command.

cat /proc/meminfo 

eg)

[prabu@dev sysctl.d]$ cat /proc/meminfo 

MemTotal:        1910704 kB

MemFree:           87880 kB

MemAvailable:     378368 kB

Buffers:               0 kB

Cached:           487368 kB

SwapCached:         8168 kB

Active:           949164 kB

Inactive:         520172 kB

Active(anon):     804996 kB

Inactive(anon):   375928 kB

Active(file):     144168 kB

Inactive(file):   144244 kB

Unevictable:           0 kB

Mlocked:               0 kB

SwapTotal:       3964924 kB

SwapFree:        3934320 kB

Dirty:               332 kB

Writeback:             0 kB

AnonPages:        974848 kB

Mapped:           123884 kB

Shmem:            198956 kB

Slab:             213228 kB

SReclaimable:     171012 kB

SUnreclaim:        42216 kB

KernelStack:        3568 kB

PageTables:        32904 kB

NFS_Unstable:          0 kB

Bounce:                0 kB

WritebackTmp:          0 kB

CommitLimit:     4920276 kB

Committed_AS:    4300400 kB

VmallocTotal:   34359738367 kB

VmallocUsed:      545312 kB

VmallocChunk:   34359187208 kB

HardwareCorrupted:     0 kB

AnonHugePages:    165888 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

DirectMap4k:       67840 kB

DirectMap2M:     1888256 kB

[prabu@dev sysctl.d]$ 

In this list , memtotal gives total installed memory (ram) ,memavailable gives the total available memory (free memory + cached memory) that can be allocated to applications.

Hugepages_Total gives number of huge pages , hugepagesize gives the size configured in my machine which is 2Mb.

Huge pages 

 In general cpu allocates memory in 4KB chunks to processes called pages ,each page has an address.

But in a 64 bit system with large amount of memory ,this approach results in a very big address space make memory management inefficient.Linux supports Huge pages where we can have pages of different sizes in addition to default 4K size, most common intel cpus support pages of size 2MB , 1GB .

A single 2MB huge page has a single address ,while a 2MB of memory using normal page will have 512 page address .So using huge pages makes memory allocation and access very fast.This is very useful for memory intensive applications like database servers etc.The only con is that huge pages require contiguous blocks of memory and it is recommended to configure huge page allocation at system startup by specifying the number of pages as below by setting a kernel parameter ,

[root@dev sysctl.d]# sysctl vm.nr_hugepages

vm.nr_hugepages = 0

[root@dev sysctl.d]# sysctl -w vm.nr_hugepages=5

vm.nr_hugepages = 5

[root@dev sysctl.d]# sysctl vm.nr_hugepages

vm.nr_hugepages = 5

Clearing memory buffers and caches in linux

Sync : 

Sync command creates a system call to kernel to flushes  disk caches in memory to the filesystems

(Both internal and external hard drives).It is similar to umount command which also flushes the caches but umount also unmounts which may not suitable for online operations.

Eg ) You have memory intensive applications open but need to poweroff the system immediately , then sync command is a crude way to help you commit changes from memory to disk without closing the applications and waiting for them to close.

Also many modern disk drive devices have write caching enabled , which cache write operations and commit the changes in bulk for throughput and power saving reasons.

Sync command has to be run as root.

#sync

Freeing  memory buffers.

Generally kernel periodically writes dirty pages to disk and manually emptying memory might slow down performance as data might be needed to read from slow disks when accessed again.
But in a server environment , where we suspect memory leaks (With long uptime ) or a need to bring down ,we can use below approach.
 We can use below command as root to free pages cached in memory , note that they will be simply discarded and we may lose data ,hence it is recommended to be used along with sync command in an atomic operation.

echo 3 > /proc/sys/vm/drop_caches

Recommended:

sync && echo 3 > /proc/sys/vm/drop_caches

The default value for the kernel parameters in most system is 0 which means pages (containing files and data)
will be cached in main memory

As per kernel.org , echo 3 applies for pages and other objects while echo 1 applies only for pages and echo2 for dentries and inodes .

To free pagecache:
 echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
 echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
 echo 3 > /proc/sys/vm/drop_caches

Dentry - Directry entries which maps file names to inodes of files , and also maps  to the filesystem hierarchy like parent and child directories.

Inode - Index nodes.In linux ,the file is identified by a number called inode. The inode also stores the groups and permisssions of a file.

Coverting a ext4 filesystem into btrfs

  I was working with ASM at my work , ASM is high performant Storage manager , Volume manager, filesystem all in one solution atop which Oracle runs its enterprise database clusters but its proprietary .I found it very elegant .

In my personal linux box the latest setup i had was a lvm (lvm2 is the  version, lvm - linux volume manager) volume group with multiple lvm volumes on top of which i had laid out ext4 filesystem (Fedora 21) and xfs filesystem (Centos 7 ) and  2 separate ntfs partitions for windows of course.

On LVM

 The advantage with LVM is we can dynamically add disks and increase size by extending the volume group over the new disk online ,extend the logical volume over the space added to volume group and resize the filesystem to utilize the added  space in the logical volume .

LVM also supports mirroring and stripping , i have not tested it much as i have been using a single disk (Its a lap) 

LVM approach is  elegant but the only inconvenience is we need to do modifications at volume manager level and filesystem level unlike asm

Btrfs Overview .

B- tree filesystem is the latest filesystem available in linux with cow, snapshots and many other new functionalities in addition to existing functionalities like journaling in ext4 filesystem .To me , it looked like the next generation filesystem i was searching for and had a design philosophy similar to ASM :)

Incidentally , Btrfs itself is initially developed by oracle which was later open sourced and merged into the linux kernel .

Btrfs has direct support for block devices (hard disks and ssds ) at filesystem level and eliminates the need for a separate volume manager like LVM (However we can still run btrfs as a filesystem above lvm using only filesystem level features , this is the example i l be covering today )

Moreover btrfs had been marked stable for more than a year and half and many distros have good enough support (/boot support may be limited based on distro ) for btrfs filesystem , so i wanted to immediately give it a try.

I did not want to reformat or backup my data and btrfs has tools for online filesystem conversion from ext3/4 to btrfs and i used it .

Below is the process to convert a non-root filesystem from ext4 to btrfs .(Please check as you may be running all your linux filesystems on a single physical partition , i had a separate /home logical volume with ext4 filer , this should not be an issue if you are using a linux distro like fedora,ubuntu,cent release in last 12 months since they by default create a lvm layout with separate mounts )

1)Install btrfs-progs if it is not present in your existing linux system,

yum install btrfs-progs

(or)

apt-get install btrfs-tools

The package manager and package name can vary based on your distro.

you can find the version of btrfs in your system using ,

mkfs.btrfs -V

2)Check the disk and filesystem lay out of your system .

You need to run fdisk command as root with the hard disk containing the linux filesystem.

eg) fdisk -l /dev/sda 

(If it is the second hard disk ,it has to be /dev/sdb )

 

 In my case it shows that i have 2 different linux LVM volume groups (fedora, cent OS ), two non-lvm linux boot partitions and some windows partitions .

Please note that when we use LVM , the ext4 /ext3 filesystems do not show up in fdisk -l output .

The lvs ,lvscan and lvdisplay commands help us view the volume groups and logical volumes in side the LVM .

 

Here cl and fedora are the volume groups created when i had installed fedora 21 and cent OS 7.

The cl volume group has 3 logical volumes home ,root and swap , similarly the fedora volume group also has 3 logical volumes.

LVSCAN gives the status of the logical volumes and the path in /dev filesystem which is a virtual system filesystem that helps us easily identify and manage devices in linux .

Lvdisplay  gives detailed information about the logical volumes .I looked up info on the /dev/fedora/home logical volume which i am gonna convert to btrfs

3) Check the integrity of the filesystems .(This is similar to when windows checks the disks during abnormal power offs )

fsck.ext4 -f /dev/fedora/home

4) Login to a non-graphical shell preferably in single user root mode and run the conversion

 btrfs-convert /dev/fedora/home 

5)Mount the converted filesystem in a separate mount point with type as btrfs

 mkdir -p /btrfs

 

 mount -t btrfs /dev/fedora/home /btrfs

Optional steps : (6 -8 )

6)During this conversion, btrfs preserves the existing ext4 metadata by creating a ext4 snapshot, we can mount it to see .

mount -t btrfs -o subvol=ext2_saved /dev/fedora/home /ext2_saved/

7) Mount the filesystem as ext4 in a separate mount point if you want to test ext4 still works

mount -t ext4 -o loop,ro /ext2_saved/image /ext4

8) In case you want to roll back/revert , unmount btrfs,ext4 snapshot and loopback image (/ext2_saved,/ext4,/btrfs) and then do it as below , (Pls note that i did not test this , I did not run revert )

btrfs-convert -r /dev/fedora/home


mount -t ext4 /dev/fedora/home /ext4

9)Remove the ext4 snapshots and use only the btrfs .

 btrfs subvol delete ext2_saved 

Check if any sub directories exist ( snapshots in btrfs are nothing but directories which can be mounted directly , you can go through btrfs architecture to have a taste of advanced features )

 btrfs subvol list -d /btrfs

 Btrfs recommends a full defragmentation and rebalance operation after a manual conversion .It will take only a few minutes.

 btrfs fi defrag -r /btrfs/
 btrfs balance start /btrfs/ 

10) Now remember that we have change the filer properties outside OS installation , so we need to update the /etc/fstab file to make the OS aware of these changes so that it mounts the /home filesystem with correct options when it boots .

If we dont update in fstab file ,while booting ,your linux distro will search for ext4 metadata in the logical volume , it will not detect btrfs metadata and throw error.

 

 

Modify the filesystem type for the filesystem you converted as btrfs , in my case i changed it from ext4 to btrfs for the /home logical volume.

Now we have a btrfs filesystem ready and usable with all existing data.Play with btrfs and share  your experiences.:)

 

How to compile the latest linux kernel on Ubuntu

How it all started

Off late , i had been feeling sluggish with ubuntu 12.10 on my laptop , a 2 year old PDC 2GB ram machine.

So first i tried switching to lubuntu 13.04 alpha then eventually moved to beta. I experienced good speedup since it removed a lot of bloatware like zeitgeist , the service that records user activity for customised search results in unity ,the desktop shell of ubuntu (Hey , i had privacy concerns too)

Now coming back to the topic , i decided to try downloading the linux kernel and compile it , i was doing it for first time. I never knew it would be this easy.

I got the 3.8.0-16 kernel source (the one already available) and tried to compile but it failed.(Need to investigate more on this)

Then i downloaded the upstream kernel 3.8.6 from kernel.org and compiled it successfully.

Prerequisite : Basic knowledge of working in linux terminal  .

The Steps 

Optional but Important. Backup any important Data
 
1. Download the latest stable kernel from kernel.org , Current stable version is 3.8.6 .It's an archive of size 70 MB.

 Link: https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.8.6.tar.xz

2.Extract it as below , (xz is a relatively new format )

xz -dv  linux-3.8.6.tar.xz

We will get a tar archive .

tar -xvf linux-3.8.6.tar

It will be extracted to linux-3.8.6 directory.

cd linux-3.8.6

3.Install the following packages using sudo or from root login,

sudo apt-get install libncurses5-dev gcc make git exuberant-ctags

I would also suggest to optionally install below packages ,

sudo apt-get install fakeroot g++ 

Actually i faced an error with gcc (when i tried compiling kernel i downloaded from ubuntu using apt-get source)

I re-installed gcc and installed g++ before trying to compile the kernel.org kernel.

4.Copy your kernel config ,

cp /boot/config-`uname -r`* .config

(We can also create a config from the one supplied with kernel , but it's a good idea to start with the one supplied with your distro , as it comes with optimisations)

5.Optionally edit the config ,

make menuconfig

In the ncurses based interface change the options you want ,I changed the processor type from pentium pro to core2 since i had a PDC P6200 CPU based on intel 2010 core micro architecture and hence wanted to compile  against an advanced micro architecture.You can also enable/disable/enable to load as module the various kernel features (eg virtualisation support . My CPU does not have virt-x extensions, so i did not enable it , curse Intel for not providing it. They are providing  it on all new Ivybridge CPU's which is good news ) at this stage.

6.Build it,

make 

or

make -jx (where x is degree of parallelism)

I used make -j2 (since i had a dual core CPU  and wanted to utilise both the cores)

Now take a break. Go for a walk , prepare tea or have a meal pr anything you like to do. Building a kernel may take a lot of time depending on your CPU/RAM and load on your machine. In took around 1 hour in my case

7.Install it,

sudo make modules_install install

It invokes the installkernel script in ubuntu which installs kernel,modules and updates grub too.

8.In case if it does not update grub (may be in older ubuntu versions),

sudo update-grub2.

Now reboot, select the new kernel , boot with it :)

 Continued..

    

        I hope you enjoyed it . It's a nice feeling to do something new and learn from it. Its a great experience , playing with linux .It just gives you more power and control which the geek in you craves for ..! 

See you again with my next experiment :)