Welcome to Linux in the Shell episode 14, the bottom of top. My name's Dan Washko, I'll be your host today and I would like to thank Hacker Public Radio for hosting this program. So if you want to contribute, check out Hacker Public Radio and do a show. They're always welcoming ideas and new hosts. So do please contribute. Now before I get going with the bottom of top, I just want to say two things. One, this is going to be a very heavy episode. There's a lot of information here to cover. So you're going to want to supplement with a visit to the website, Linux in the Shell.org and looking up at the episode 14 because we're going to be covering a lot, a lot of content because the bottom of top is very configurable and there's a lot of stuff. So without further ado, let's go ahead and get rolling. Now last week, we're last episode, I should say, we talked about the top of top. Top is broken up into two different sections. The top five lines or so by default is a summary area. And that shows you your CPU memory information and general information. So check out the episode 13 for the full skinny on there. We're going to be talking about the rest of the top interface and that's the bulk of it. And that's everything underneath that by default gray bar with items in it. And that's called the task area. That is essentially a list of all the processes running on your system. And it's going to be ordered in, in any manner that you will you choose. By default, it is ordered by CPU utilization and we'll get to that or CPU percentage and we'll get to that in a minute. What I'm going to do is I'm going to go through the columns, default columns at the top that you'll see, a little explanation of those. And then I'll talk about the other columns that you can add with some explanations and then how you can configure that. So let's jump right in. When you start off top, default window is going to show to is going to show a column in a gray bar and that's going to have the field names. And then each process underneath that is a row with the information filled in accordingly to each field name. The first column is going to be called pit. Pit is the process ID. It's the unique idea associated with the process information that's detailed on that row. So that's going to be your pit right there and then next to that is going to be user and that's the user account the process is running under. It's the owner of the process. That's followed by PR and that stands for priority. Now this gets a little complicated because what's being reported here is the schedule priority of the task running. And that value is dynamically generated by the kernel using the nice value. And the range is pretty dynamic. But there is a calculation and what I found the calculation is N i which is nice plus 20 minus x to N i plus 20 plus x. So the value is a range that an x in this is a bonus or a quote discount point. So the range is dynamic and it can be a negative number or a positive number. And these these values are adjusted over time. And it's dependent on how the process utilizes CPU time. Now for process is sleeping a lot. It'll adjust the x value of that process. So it decrements. Now for process uses a lot of CPU time. It has their points adjusted or that x value. So the value increments. So when the scheduler checks to see what it process to run, the process with the lowest priority or the lowest number will run first when the schedule looks through it all the processes. Now you probably see many of these processes running have the same PR value if you're looking at top. And that's kind of to be expected. Most of those you will see with the same PR level between 20 and in 1920 or whatever are probably sleeping. Now there's another value in there. It's RT and that stands for real time. So be aware of that. It's a value that's determined by the dynamically by the kernel. So that if a process is sleeping, it's probably waiting for some IO possibly or whatever, that it's possible that the kernel will start adjusting the value. So the PR value lowers and gets negative. So when the scheduler comes through and checks to see what process it needs to run when it's free, it'll choose one with a lower value that's ready to be run. So that's a way for the kernel to say, hey, this needs to get taken care of when it's ready right away or this isn't necessarily a high priority when the scheduler's free. So that's what PR stands for, scheduling priority. Next to that is the nice value of the task. The nice value is a number from negative 20 to 19, where the lower value has the highest priority. So something that has a negative 5 has a higher priority than something that has a value of 10. Now most processes start off with the value of 0. So just write in the middle and that's what you'll see a lot of these processes running under. Unless the nice value has been changed, but otherwise you can see on there what the nice value of the process is running at. The next column is Virt virtual memory size. Now this is the total amount of virtual memory used by the process. And it's not necessarily real memory used in a physical memory, but it includes data swapped out the disk or cache shared libraries, etc. So don't miss, don't confuse this value with the physical RAM value. And that is the next column which is resident memory size. Resident memory size is how much physical non-swappable memory the process is used. So that's a more accurate representation of how much physical RAM that process is taking up. And you'll see after these values more than likely an M if it's megabytes or G if it's gigabytes or if it's just I believe K if it's kilobytes and if it's just bytes you'll see nothing listed there. But that'll give you an idea of how much memory that process is running utilizing. So virtual memory size is amount of virtual memory used by the process where as RES and resident memory size is how much physical memory is being utilized non-swappable memory. Then there's SHR which is shared memory size. Now that's the amount of memory available to a task that is shared with other processes. Now for instance if you had multiple instances of an application like a bash shell running instead of each one firing up and using you know the same resources and memory over and over duplicating those resources the operating system will allow for those same libraries to be shared. So where it can be shared it will share those values instead of each individual resource taking up its own bit of memory it can share that memory between them. And this will show you how much memory of that. How much of that memory is being used by that application. It's the library set is using the shared libraries and overall load on the system. So that number might be a value between what you're seeing in virtual or shared or it might be you know significantly lower. So shared memory is how much of the resources is taking that are being shared between other applications. S column stands for the status and this is a one of five values you're going to see a D which is uninterruptable sleep and S for sleeping. Now what's the difference between uninterruptable sleep and sleeping? Well a process that is running as sleeping can be interrupted by a signal. But a process that is uninterrupted will sleep cannot be interrupted. Now what that usually means is a process as a status of D or uninterruptable sleep is waiting for a resource to become available like a disk. So any signal sent to a process in the uninterruptable seats sleep state will accumulate until that and be handled when the process returns from the sleeping state. So it'll queue up those signals that might get sent to it if that's the case. And instead of interrupting the sleeping process the process those signals it'll wait till whatever that process is waiting for completes it runs and then it'll process those those signals that were sent to it. So that's the difference between uninterruptable sleep and uninterrupted sleep or sleeping in general. Then there's running. Running is means the process is probably running or it's ready to run. So that that is good to go. Then there's T and T stands for Trace or a stopped process. A process that might have been stopped by hitting Control Z. And then finally there's Z for Zombie, which means the process is has issued a that is finished but it's waiting for the signal from its parent to catch up and clean up after it. Last week or last episode, towards the end of the episode I had noticed that there was a zombie process running. And I had said I wonder what that zombie process is and I couldn't figure it out at the time but I then figured out what it was during the screen capture session that I was running for the video. And that was the GTK record my desktop has spun off the process that was a zombie process and that cleaned itself up when I was done with the screen capture. So zombie processes are not necessarily bad so to say or a cause of a problem. It could be the way the program is written. It's just waiting for something to complete to get back to it. But if you see zombie processes sticking around stacking up that could be an indication of the problem but we talked about that last episode. Then there's Percent CPU and that's the task share of the CPU time utilized since the last refresh. So if you're running in a multi-processing environment, remember that the default Iris mode is on and the value percentage is a percentage of the combined CPUs. So in a sense if you have two CPUs and you see a percentage of the CPU utilized since the last refresh is 10%. That's 10% of two CPUs like 10% of 200%. So be aware of that. Whereas if you turn off Iris mode which I explained last episode, you end the Solaris mode and the values the percentage divided among the total number of CPUs. So in that case, the 20% would be 100% of both CPUs. If you switch between the mode, you're not going to suddenly see 20% and Iris mode drop down the 10% in Solaris mode. It does not that clean cup but essentially that's the way those two modes operate and for more information consult the last episode and the notes there in. And refresh rate, remember, by default, is three seconds but you can adjust that if you want to. Now percent mem is the amount of physical system memory used by the process displayed as a percentage and that's going to show you the percentage of physical RAM used by the process. Then you'll see time plus. Time plus displays the total CPU time the task is utilized since it started in hundreds of a second. Now what this means is if you look at it, I mean since this process started, if you're looking at it, it has consumed this amount of the CPU, this amount of CPU time. So you might look at a process that you've had up and running for a bit like right now I'm recording this in audacity and it's pretty CPU intensive. Well, not very CPU intensive but it uses a lot of the CPU. It's going at Cp 11% CPU utilization since the last refresh, fluctuates between those. Remember, I'm on iRix mode by default. So this is a dual core system. So that's between 11 and 13 or 14% of 200% that's being used. Now if I switch it between iRix mode, turning off iRix mode and turning it on, it changes those values but they don't they don't differ that drastically. So just just be aware of that anyway. Back to what I was saying now the total CPU time since I started this application and it's being used is now about two minutes of CPU time that's being utilized. It shows one minute 55 seconds and like 0.500 of the second. So it puts it down there and shows you how much CPU time that resource has consumed. The plus means it's allowing hundreds of a second. Now there's another two options that you can have going on here. The default cumulative option which the value is off, it does not include any of the processes, dead children. So if a process spun off a child process and then the child process of course finished, it's considered a dead child, very morbid of course. Cumulative value is off so it doesn't show that those values in there but if you turn cumulative mode on by pressing capital S, it'll show you the value of any dead children that were spun off by that process included into the time of the CPU was being utilized. Finally we have the last line by default is the command and that's pretty simple. The command that is the process is running under the name of the command. You can toggle this to the command line by pressing the lower KC and that'll show you the command that was executed on the command line to generate that process. If you see a process that wasn't in that case is in brackets that means that the process wasn't started from the command line like a kernel thread and so that that value is going to be contained in brackets. Now you might say well I didn't start the process from a command line. I started it from a run line in a window manager or desktop environment are clicked on an icon. Well essentially it's the same starting from the command line and many regards because if you pull that up you will see. Okay so that covers the default columns that you can have in the top window. The next list that I'm going to talk about or the optional value said you can easily toggle on and off and I'll tell you how to do that when it's finished covering it so I want you to remember that most importantly. Okay now this is a fairly long list so strap yourselves in because here we go we're already at a good 17 minutes and this is the lion share of the show. C group okay this column option lists the control groups that the process belongs to. Now for process doesn't belong to a control group a dash will be displayed. Now what a control group is the feature of the Linux kernel to limit account and isolate resource usage of process groups. So control group is a collection of processes that are bound by the same criteria and that's kind of what the definition is. There are tools to create control groups like CG create CG execute or CG classify and they are especially useful and virtualized environments to help ensure that one group or program does not exceed the resources allocated and a pair system functionality for other users or processes. It allows you control groups allow you to define resources that specific group can use and even limit access to a specific resource if need be. Control groups are organized hierarchically where children control groups and hair attributes from the parent. SNN and nutshell with a C group does it's a little beyond the scope of the intent of this show right now to go in the further detail about C groups but if you really want to know more head on over to the website I have links in the show notes to give you more information about those. Code is an option and that displays the code sizes the amount of physical memory devoted to the executable code in kilobytes. This is also known as the text resident set by some other applications like PS. Now this value shows how much physical memories actually being used and it excludes what swapped out. So that's code. Then we have data. The data entry details the amount of physical memory used by the process that is devoted to everything but the code. So that would be like you know in a VI if you're running a VIM session the code that VI runs under would not be including this section you would actually be what the text is or swapped out to like to drive or if that's in resident memory how much of data is being utilized that's not code. And we have flags and flags as a hexadec, hexadecimal representation of the tasks or and schedule flags and zeros are suppressed. That one is a difficult one to define. I'm not going to try and cover that right now because I spent a lot of time digging through different things. There's going to be a list of hexadecimal values for all the flags that are appropriate to with the task is running under by the kernel and if you if you go in there they say it's covered under the uh how do I say it's covered under the includes file if you start reading the includes file that's uh trying to remember what the name of the include file is. I tried digging through this one and this one was tough. This one was really tough to go. It's possibly in the uh like user include Linux slash schedule.h and they say look in there the flags are officially documented in there. I did not see all the flags in there and I looked all over the web and stuff so I have some resources that made that that talk about that. It's a fairly complex topic. Go to the notes if you want more information but that'll show you the hexadecimal representations of the scheduling flags for that task. Then there's a GID which is the group ID the process is running under or you can choose group which is the name of the group that the process is running under instead of just showing you the GID. Then we have something called NDRT and this is the the count of dirty pages and those are pages that have been written to auxiliary storage. So when operating system needs to bring a page into memory and if there's no physical page free what the OS will do is attempt the discard pages that are not in use in the physical memory. Now a dirty page in this case is uh data that's in memory that has been altered but not saved disk so the page really can't be deleted and as it may be need to be called again so it must be saved out to like a swap file. So if you're like working on a process or if you're working on a text editor this is kind of high level looking at it and you're editing something in the pages resident and memory the values are in memory but you start going on doing other processes need to be and that starts to sit idle there's a chance that if the system needs to swap out to free up some pages it might look at that and say well here's information a page that has been altered it's not the same that's on disk but I can't dump it out of memory because it hasn't been saved I need to move it over here to swap area until to handle this new bit of information and he's going to a page and if it gets called again I could pull it out of the swap but it hasn't actually been committed to any auxiliary storage and we have NMAG and that's the number of major page phones that have occurred for a task. Now when a process attempts to read or write to a virtual page that is not in its address space that's a page full okay what makes it a major page file is when auxiliary storage access is involved and making that page available it is five as a major file so if it's trying to access a memory space that's on auxiliary storage not in main memory and it's not in its address space that's a major page phone whereas then there's NMAG which is minor page file is the same thing it's trying to access address space that's not in this current address space but it's not stored on auxiliary storage it's resident memory but we have NTH now this column shows the number of threads associated with the process so how many threads a process may have spawned or that are associated with that process. P stands for the last use processor in a multiprocessor system if you only run a one processor it's probably just going to show you one but what it'll do is it'll show you what was the processor that the task was running on at that time of the last refresh then we have PGRP the process group ID processes are groups in unique groups for the distribution of signals and terminals and by terminals to arbitrate requests for their input and output it's taken from the Wikipedia page I believe a child process our members of their parent groups so when a new process is started the process group ID is usually set to the process ID and becomes a group leader if it's a new process but children usually inherit the process group ID of the parent so a lot of times you'll see PRGPID set the zero which is the in it process group ID which is the process ID of the in it application when run so in it's running under a PRPGRP of zero so a lot of processes you might see in top running with a process group ID of zero and that's because they've been spawned from in it whereas if you're running a new app and I could pull up a terminal and you fire off an application it might generate its own process group ID depending on the application will might inherit it from the terminals PPI ID this column represents the parent process idea of the process more often than not you'll see a lot of processes is having in it parent process ID so this this will just show you a process spawned from a parent or what the parent ID might be it will show you that process ID are you ID stands for real user ID that's the real user ID the process is running under generally the user who started the process now this is different than the effective user ID and at the effective user ID can be different from the real user ID if the process is running under that the process that it's running under has been altered using a command like SUID to run that process so if you were to run a process using SUID command the effective user ID may be root or somebody else but the process real user ID is going to show up is you because you were the one who started it similar to that is the R user which is the real user ID I mean real user name sorry instead of showing you the ID that will show you the real user name the process was started under we have sit sit is similar to process group ID but it's the session ID that the process is a member of the session ID is a collection of process groups that is usually started by the login shell so that will show you the session ID then we have SUID which I had mentioned before now SUID stands for saved user ID so when a program is running as a privileged user and it needs execute commands as an unfrivileged user what it does is copies the privilege user ID to the SUID value so this is what's reported by top so it knows when it needs to return to the elevated privilege user that's the SUID or the elevated privilege user ID that it needs to run under now like our user or our our UID and our user SUID has an SU user and that just displays the name instead of the ID oh I forgot one I forgot a couple missing all this here SUP GIDs this is a column contains the IDs of any supplementary groups the process is running under and of course there's SUP GRPS and instead of showing you the ID it'll show you the group names separated by column commas we have swaps and this is the find as the non-resident portrait portion of a task's address space so the amount of address space the task is using that is not resident memory so stuff that might be swapped out to swap by our cached cache file or whatever we have TGID that's a thread group ID it's more useful with a multi-threaded process because a single threaded process will report the process ID so if it's a multi-threaded process it'll show you the group ID that process is currently a part of we have time which is a little different than time plus I mean this is the same thing it's a total CPU time the process is used since it started but the values and seconds here are not hundreds of a second and of course you can toggle on and off cumulative mode with the sq lowercase s TPGID TPGID is the process group ID of the foreground process for any connected terminal that started the process if the process is not connected to a terminal the value the negative one is given so it shows you what the process idea of the terminal that it was pretty much what it was the process idea the terminal that it's connected to TTY is the name of the terminal controlling the process is generally this is going to be a device name you ID the effective user ID of the process or you can have user which is the effective user name the process is running under we have we have WChamp right now with WChamp is this column shows the name or address of the kernel function in which task is currently sleeping at the process is not sleeping then it will display a dash instead of an ID or address space right there or name so that some of that stuff is probably really only helpful to programmers or debugging issues on their application but if you want more information on a lot of that head on over to the website and look at the notes there's a lot of stuff to cover there give you a lot of information so then in nutshell I've given you the default values and the values that you can toggle on and off in the display of top now you might be asking yourself Dan how do I do that well it's very very simple you press the lower case fq just press the fq it's lower case and that will switch to top window that you're looking at to a new window called the fields management window and right at the top of the fields management window is an explanation of how to navigate this window so what you're going to get is directions at the top then you're going to get a list of all the columns that you can toggle on and off with more than likely if you're not changing anything the defaults first and then the rest of them now what you'll see are right off the bat you will see ones that are bold that are highlighted in a default window there'll be just like bolded white and then you'll see just normal colored text on these values so what you can do to navigate this what the bolder means is that those are the values that are currently displaying in the fields those are the fields that are currently displaying and you can they'll have an asterisk next to them too and you can navigate this list by pressing the up and down arrow keys and you can move up and down and if you want to toggle a field on or off all you have to do is hit the space key or the d key for display and if it's it's bolded and displaying it will it will turn that field off if it's not bolded it will turn that field on and you will see it become bolded and put an asterisk next to it also what it will do when you do that as you're moving up and down it'll allow you to adjust the position of one of those fields that's very simple so you're using up and down the navigate d or s or space to toggle a field on and off then you're going to use the right and left arrow keys to move one of those columns around so you select it the field by pressing the right arrow key and then you move it up and down to where you want it to be and the list of displayed values then hit the left arrow key to set it there now of course you can move around you can move the value around the list and between the non like if if you had all your fields at the top the fields at the top are bolded like the first 15 bolded and then the remainder are on bolded or not they're not active and you take one of the bolder ones and move it down below an on active one all it's going to do is put that that field to the end it's going to move it to the end doesn't really impact on what fields are not displayed and how that order is unless you turn one of those fields on before it or after it and then it will display it in those orders so it's very simple up and down navigates this field list s space or d toggles whether the field is displayed right arrow key selects one of the fields for moving or changing its position and up and down again to move where you want to and either left arrow key or enter commits that change so when you when you're done all you have to do is hit escape or cue and you will see your changes made in the task area of top and you'll see that that new column has been added or an existing column has been removed or some things may have been adjusted in other locations that my friends is the bottom of top in a nutshell a mouthful there's a lot of stuff to cover there might have been a little dry in some respects but there's a whole lot of information that you can gardener from top and give you a snapshot of what's going on in your system remember you can always change refresh rate when you're in the top window by pressing the d key and it'll change to delay and I think you do up tens of a second so like 3.5 we'll give you 3.5 seconds default this 3 pretty handy application now there's still more to talk about in top so the next episode is going to focus on different views of top how to configure that stuff and any other stuff that I actually didn't mention already about top we're going to wrap it up on the next episode so I do hope that you will join me again in two weeks for the rest of top and the meantime head on over to the website watch the video read the show entry to further solidify this information in your mind and check out the notes if you want some further information on any of those fields again my name is Dan Washko you're listening to Linux in the show hosted by hacker public radio thank you very much have a great day you have been listening to hacker public radio where techer public radio does are we are a community podcast network that release the shows every week day you won't have to Friday today show like all our shows was contributed by a HBR listener like yourself if you ever consider recording a podcast then visit our website to find out how easy it really is hacker public radio is found by the digital dot pound and the economic and computer cloud HBR is funded by the binary revolution at bingreff.com all bingreff projects are produce sponsored by lunar pages from shared hosting to custom private clouds go to lunar pages.com for all your hosting needs on this otherwise stages today's show is released on your creative comments at reviews show share a like details are all licensed