25 Jul 2014

Links and metacharacters in Linux

Leave a Comment
Links and meta characters in Linux   We will continue our Linux journey by talking about two important components of the File System:
links (soft and hard links) 
metacharacters
This would be an important step to further understand how this OS works and what are the main concepts behind its functionality. I will try as best as I can to explain these two concepts so please post a comment if you think that things were not described properly.
   Besides the actual data contained in each file within the File System, the Operating System will also store specific information for each one. Files are described by multiple attributes and we've seen some of them when we've learned about the ls -la command in this article. Attributes like file type, group and user ID, file permissions, file size, last modification date, creation date, etc, must be stored somewhere in the File System. In Linux, all this information is stored within inodes (Index-nodes). An inode will keep all data regarding a particular file except its name and each one will be identified by a unique number. They also store a list with all data blocks that are part of the same file and where is each block located on the hard disk. We will later see how you can view and interact with inodes. Basically, for every file there will be an associated inode allocated.

   A Linux partition is composed of several elements. In the following section I will try to explain the role of each component:
boot block - the location of the boot loader. We've described the Linux bootloader in a previous article.
super block - contains information like number of data blocks, mount-points, File system type, Size, Status, Information about other metadata structures, etc.
inode list - the list of inodes contained by a partition. Note that this is a finite number.
data blocks - this is where file content is stored. A file can contain one ore more data blocks and inodes are the ones who store the information about where is each data block located.

   In Linux, one file can be seen in multiple locations without multiplying its content. Practically, you can view the same resource in multiple locations but, all will have the same source file. This feature is handled in Linux using links. When talking about links in the UNIX/Linux Systems, we'll need to specify the two concepts behind this technology:

soft link also known as symbolic link - it's a redirection from one file to another. They have similar functionality as Windows shortcuts. To create a new symbolic link you'll need to use the following command:
ln -s existing_file new_file
This command will create a new "shortcut" to the file that already exists in the File System.
hard link - an association between file name and inode. As we've talked previously, the name information is not stored in the inode but, instead it's located in the directory which contains the file. If you want to create a new hard link for an existing file you'll need to use the following command: 
ln existing_file new_file
The command will basically create a new association which will contain another name and the existing inode.
To view the number of hard links for a file you can use the ls -l command (we've talked about this in a previous article)

Metacharacters are mainly used for filtering the output of a command. By parsing information before manipulating a command's result, offers System Administrators a powerful way to manipulate data within the File System. You will see that there are special characters reserved in the OS which have a distinct functionality and can often be used to simplify administrative operations. Regex (regular expressions) are build using metacharacters so, this part may sound familiar. I will try to explain the role of each metacharacter within the OS, please add more if you think are relevant on this topic:

> Used to redirect the output of a command to file. If the file does not exist, it will create a new one, if the file already exists its content will be replaced
>> Redirects the output of a command but, unlike the > character, it will append the new results to an existing file.
< used to redirect input
* wildcard used to substitute 0 or more characters. Example ls -al f* will output any file which begins with f
wildcard used to substitute only one character. Example: ls -al f??e will output any file that starts with "f", contains any two characters and then ends with "e"
[ ] will substitute any character between brackets. Example: ls -al f[iad]e will output fie fae fde
^ any other character except the ones indicated when used between []. Example: ls -al f[^a-c]e will not output fae, fbe ,fce
; used to separate multiple commands that are written on the same line
Predefined metacharacter class:
[[:alnum:]] - alphanumeric characters
[[:alpha:]] - alphabetic characters
[[:blank:]] - SPACE or TAB keys
[[:digit:]] - numeric characters
[[:upper:]] - upper letters
[[:lower:]] - lower letters
|| OR conditional execution
&& AND conditional execution
# used to specify comments within files
\ if you use this character in front of any other metacharacter, the output will use that particular symbol and not its special functionality. Example: 300\$ = 300$
. filters one character. Example d.g = dog, dag, deg
"$    preceeds the name of a variable to be expanded.
The variables are either assigned using  "="  or are one of the pre-defined variables (which cannot be assigned to):
$0  name of the shell or the shell script being executed.
$# number of the positional parameters to the command
$1  the value of the first positional parameter passed to the command. $2 is the second positional parameter passed to the command. etc. up to $9.
$*    expands to all positional parameters passed to the command
$@   expands to all positional parameters passed to the command, but individually quoted when "$@" is used. " from About.com
I've found this interesting article which describes regular expressions and metacharacters you can read it to discover more information on this topic. You can also use man bash command to discover more things about metacharacters.
That's about it for this article folks, please share your thoughts about it and don't forget to share it to others. Enjoy this wonderful day and stay tuned for the following articles from IT training day.
Read More
24 Jul 2014

Linux user administration

Leave a Comment

Linux user administration tutorialIn today's article we will talk about user administration in UNIX/Linux distributions. If you are familiar with Windows user administration then, this lesson will be much easier to digest. A Linux account is composed of several components: username, password, home directory and environment. You'll need to know that each username is identified by an UID (User ID) which is a unique number in the users database. The correspondence between UID and username is made to make things easier for System Administrators to identify System users.
   Users are organized into Groups to allow multiple accounts to interact with resources within the Linux File System. Each group is identified by a unique GID (group ID). Remember that each user can be part of one primary group and up to 15 secondary groups
   Just like in Windows, each file will have permissions assigned for several entities. In Linux, these entities are: owner, group and others. It's important to understand that each file will have these three permission categories assigned. Further more, each process running in Linux is executed using an UID and GID which basically sets the process permissions within the OS.
   The UID is a unique number that is below or equal to 2147483647. UID can be divided into three categories:
0 - super user account (root)
100, 500 or 1000 - depending on the distribution standard services and applications will require a service user to execute them. Remember that these accounts will normally not support interactive login and are used to serve as service accounts only.
above 1000 - Standard user accounts

There are several files which are responsible for storing the username, password and group information:
Linux passwd file/etc/passwd – username database
/etc/shadow – users password database
/etc/group – group database
/etc/gshadow – group password database
You can view the contents of any of these files by executing the cat command. Let's take for example passwd file which stores the System usernames: cat /etc/passwd

Each database entry is composed of several elements:
username:x:UID:GID:comment:home_directory:login_shell
  • x - password field used in older Linux distributions, System passwords are now stored in /etc/shadow database
  • comment - used to add a description for a particular user
  • login_shell - default shell used by users when interacting with the Operating System.
Let's take a look in the password file using the same command: cat /etc/shadow
Each entry in the database is composed of several items:
loginID:password:lastchg:min:max:warn:inactive:expire:
    Linux shadow file
  • loginID - username which corresponds to an entry in the passwd file
  • password - password hash
  • latchg - the last date when the password was changed
  • the min, max values are the minimum and maximum days that must pass before the password can/must be changed
  • warn - a warning message can be set before the password expires
  • inactive - number of inactive days before the account is locked
  • expire - account expiration date
Linux group file
The group database includes secondary groups for each account. Type cat /etc/group to visualize its content.

As you can see from the command's output, each entry is composed of the following fields:
groupname:group-password:GID:username-list
The username-list contains users that are part of that particular group.

Below you will find the main commands used to interact with usernames, groups and passwords
useradd,userdel,usermod – add/delete/modify users
groupadd, groupdel, groupmod – add/delete/modify groups
passwd – tool to change user's passwords
The following commands are used to interact with specific fields from the shadow and passwd databases: chfnchpasswdchagechsh

Useradd - command supports multiple parameters, I will point out some of these parameters:
-s - default shell used by the user
-c - additional comment can be set
-g - primary group of the user

-G - secondary groups

-u - sets the user's UID
-d - specifies the personal directory of the user
-m - creates the home directory
-U - creates a group that has the same name
Example: useradd -u 2014 -d /home/user1 -m -s /bin/bash -U user1
To explore all parameters for this command type man useradd or useradd --help:
Useradd command

Linux useradd command
We can set default parameters used when executing useradd command by editing the /etc/default/useradd file. You can used the -D parameter to visualize and interact with the default parameters used with this command. The command also supports so called skeleton directory which contain default configuration file for the user's environment. The skeleton directory can be set using SKEL=/etc/skel or useradd -k /path/to/skel. You can configure standard configuration files for different applications: bash,

.bashrc, .bash_history, .bash_profile, .inputrc, .vimrc, etc.


The usermod command supports similar parameters as useradd and it's used to change different fields from the username database. It would be good to remember the following parameters:
Usermod command
-U - unlock user account.
-L - lock user account. When a user is locked, a "!" symbol will be placed before its password in the /etc/shadow file. When a user is unlocked, the "!" character disappears:
Linux Usermod command
Linux passwd command

To change a user's password type passwd username. Let's take the newly created user and set a new password: passwd user1


Linux Userdel command
To delete an existing username, execute the following command: userdel username. The command with any parameters will delete the user from the passwd and shadow files but, will not delete its home directory. To include the home directory deletion in the command execution, add the -r parameter

That's about it for this article folks, please share your thoughts about it and don't forget to share it to others. If you have any suggestions regarding this topic don't hesitate to use my comments dedicated section. Wish you all the best and don't forget to enjoy your day!
Read More
15 Jul 2014

How to install and configure Windows Deployment Services

Leave a Comment
   Microsoft Windows Server 2008 offers a possibility for automatic deployment of Servers and Workstations. This technology is easily configurable and can suite most organisations in terms of Windows  Server deployment. In this article we will see what are the components that make up WDS ( Windows Deployment Services ) and we will learn how to successfully install and configure this role. When installing a Windows Server using the Installation Wizard, the Administrator must configure several parameters; with WDS, all these settings can be configured automatically using so called answer files. These are XML file s which contain all settings that a machine must have upon deployment. When the Server Installation process is executed, the wizard will look for the answer file on one of its partitions. The answer file is called autounattended.xml.
   One requirement of using Windows Deployment Services is that deployed servers must have a network card which supports PXE. "The Preboot eXecution Environment (PXE, also known as Pre-Execution Environment; sometimes pronounced "pixie") is an environment to boot computers using a network interface independently of data storage devices (like hard disks) or installed operating systems." You can find out more information about PXE on Wikipedia.
   When the machine is deployed, the PXE network card will automatically discover any available WDS Server and will initiate the Installation process. WDS allows you to store both System image files and answer files thus the whole deployment process is made from a single location. Deployments can be made using either unicast or multicast technologies. Multicast is recommended when deployments are made on multiple machine at the same time.
   Before installing WDS on a Windows Server, you will need to make sure the following requirements are met. First, ensure that there is enough space to store the Image files. Another requirement is that you'll need to host all Servers within Active Directory and also make sure there is a functional DHCP server on the network.
   WDS uses Installation images when deploying servers. These images contain the files needed for installing and configuring Windows Server 2008; we will see how to configure Installation images. Note that depending on the Server's architecture you will need to add Installation images for X64 and/or X86 architectures so you can support them within your network. WDS requires boot images which are used to boot a Server before the actual installation process. With a boot image, and Administrator can choose what Installation file will be used on the deployed Server. For computers that do not support PXE, WDS offers the possibility of using so called Discover Images which are deployed using DVDs or USB devices. A boot image can be created using the capture utility within WDS. This tool will allow Administrators to create an System Image from a Server that was previously deployed. This way, you can create reference images for all Servers ensuring that all machines will follow the same standards and are configured properly.
How to install Windows Deployment Services   We will now proceed to the Windows Deployment Service Installation. Open up the Server Manager Console, navigate to the Roles section and click on Add Roles button. From the available roles list, select Windows Deployment Service and proceed to the next section. Note that I'm using a Virtual Machine that hosts ADDS, DNS and DHCP services. You will need these services configured before installing WDS:
How to install Windows Deployment ServicesMake sure that both Deployment and Transport Server are checked and proceed with the installation:
Configure Windows Deployment ServicesOnce the setup is completed, navigate to Administrative Tools and open Windows Deployment Services console. Expand the Servers section, right click your Server's name and select Configure Server. The Server that has WDS installed is automatically added in the console. 
Windows Deployment Services WizardIn the following section you will need to select the remote installation folder. Under this path you will store the installation and boot image files needed in the deployment process. Note that the selected partition cannot be the System partition and you must use other NTFS partition. I'm using a VM with only one partition so, I will leave the default settings:
Windows Deployment Services Wizard Configuration WizardI'm using the same machine for hosting WDS and DHCP this is why I will need to configure the WDS Service not to listen on port 67 (DHCP conflict):
WDS Configuration WizardWe will allow our WDS Server to respond only to known client computers. This means that each Server must be pre-added to Active Directory before the actual deployment process:
WDS ConfigurationWait until the configuration process is completed then check the Add images to the server now setting and click Finish:
WDS Add Image WizardYou will need to select the path to your Installation DVD/Image files. This image will be used when deploying a new machine:
WDS Add Image WizardIn the next section, make sure that the Create an image group named option is selected and add a name for your group:
Windows Deployment Services Add Image WizardIn the final Review Settings section, you can view the number of installation and boot images that will be transferred on your WDS Server:
Windows Deployment Services Add Image WizardThe operation will take some time depending on the size of your installation images:
Note that in WDS, you can store multiple installation images from different Operating Systems like Windows Server 2008, Windows Server 2012, etc. You will add each image using the same process.
WDS Multicast TransmissionReturn to the WDS console, navigate to the Multicast Transmission section, right click it and select Create Multicast Transmission:
Create Multicast Transmission WizardThe Wizard will prompt you to type the name that will identify the multicast transmission. Next, select the newly created Image group. In the following section, we'll select enable the server to start transmission when a client has requested the image:
Your WDS Server has now been configured to deploy Windows Servers. In the following article I will show you how to deploy a new Server within your network using WDS. Several network requirements will have to be met before we can proceed with the deployment process but, we will cover all aspects in the next post. Hope you've understood how to install and configure WDS, for any misunderstandings don't hesitate to leave a comment. Wish you all the best and stay tuned for the following articles from IT training day.
Read More
14 Jul 2014

Linux shell commands to Interact with the File System

2 comments
Hello dear readers,
   Now that we've got acquainted with Linux Operating System and we've seen how to partition our hard drive and what is the basic structure of the File System, it's time to further discover this amazing OS. In today's article we will learn about some of the most important shell commands that are used to navigate and interact with the Linux File System. Most commands described in this article may be already known by most of you but, I'll write and explain them for any novice.
   When navigating in the Linux File System you will need to know the paths used to gain access to resources. You can imagine the File System as a tree with several branches. That being said, there are three ways to describe a path to a file:

absolute - paths that start with root (/) and describe the complete path to a resource. For example /var/www/icons/a.gif. When you interact with a file, its absolute path is loaded in RAM so any command executed will apply it to the file's absolute path. For example, if I type cat a.gif when I'm in the icons directory, the OS will actually execute cat /var/www/icons/a.gif

relative - they have as reference the current directory. There are two symbols used to describe a relative path:
    Linux tail command
  • .  - it's used to describe the current directory. For example, if I navigate to /usr/bin, I can execute tail command by typing ./tail. This actually tells the OS to execute the command from the current directory.
  • Linux File System .. - refers to the parent directory. Let's take our last example: the absolute path to our current location in the File System is /usr/bin/tail. The absolute path to one of my user home directory is /home/dan. If I want to change the path to the user's personal directory from my current location, I would type cd ../../home/dan. So basically this command means go back two parent folders (/bin and /usr) and then navigate to /home/dan. Simply put, ".." refers to the parent directory:
Linux tilde character~ (tilde character) - from any directory you can change location or make reference to a file from your personal /home directory using this character. For example, the absolute path to my /Desktop directory would be /home/dan/Desktop but, I can simply use ~/Desktop to refer to this location.

Navigation commands used within Linux:
cd (change directory) - used to jump from one location to another. This command is also available in Windows Systems (Batch and Powershell) and has the same functionality:
Linux Change Directory command

pwd - displays the absolute path of the current working directory:
pwd command

Listing files and directories:
Linux ls commandls - used to display files and directories from the current directory. You can specify relative and absolute paths when using this command. Note that ls can be used with multiple arguments. You can view the help page by typing ls --help or man ls to view its man page. Without any arguments, ls will display files and directories from the current location:

I usually use ls with -la arguments. Remember that in Linux, command arguments can be written individually or combined ( ls -a -l = ls -al):
-a - display all files ("hidden files"). Config files usually start with .(dot) and are not shown by default.
-l - long listing. Displays all info relevant to a specified resource. Let's take the following example and explain what each section describes. I've changed the working directory to /etc and typed ls -la
ls with long listing command in Linux

As you see from the command's output, there are several sections for each file. Let's take bashrc file and explain each section:
Linux File System tutorial
  1. - informs the user that this is a file, for directories you can see there is a "d" in front of the line
  2. there are 9 characters that describe system permissions (rw-r--r--). we will discuss about permissions in a future article
  3. 1 is the number of hard links that point to this file. We will learn about hard links in a future article
  4. the first root is the owner of the file
  5. second root is the group that has ownership on the file
  6. 2681 - file size in bits. We can add another argument to our command to make the file size easier to read. Type ls -alh where -h stands for human readable and you will see that the sizes are displayed in kilo. Note that for directories, the command will display the directory file size and not the size of all files contained in the directory
  7. 2013 - last modification date
  8. file name 
ls recursive command
   Another parameter that you can use with ls command is -R (recursive). This argument displays each file and directory and their containing files and directories and so on.

   Other cool parameters that can be used with ls command are -d, -i but, I'll let you folks discover their usage. Explore the help menu and the man page and you will be able to master the ls command :)



Interacting with disk and file size
du (disk usage) - used to view file size in a particular location. Can be used with the -h parameter to view sizes in kilo. You can either use it on a file or on a folder; when executing the command on a folder, it will display the size of all files and directories contained:
Disk usage command Linux

Linux du command
   -c adds the total size of all files at the end of the output ; -s displays the size of the argument passed to the command so, for a directory, will display its total size. I've combined these two parameters to see that they have the same output.

Linux Disk free commanddf (disk free) - Displays all disk information from all mounted devices (hard drives, external shares, etc.). Can be used with the -h parameter to have a better output of the command:

Create and remove files and directories
mkdir - creates directories in the specified path. Can be used to create multiple folders at the same time. -p parameter can be used to create a whole path to a specified resource:
mkdir command

touch - used to create files within the Linux File System. You can specify multiple files to be created from one execution of the command. For example:
touch command

Linux rm command
rm - removes file and directories. Can be used with the following parameters: -i - prompts with every removal; -r - recursively delete, used to delete directories; -f - forces the removal of a directory

Copy and move files and directories
Linux copy commandcp  - copy files and directories from one location to another. You can specify one ore more arguments and you can copy multiple files and directories at the same time. For example: cp f4 f10 ; cp f1 f2 file1/  . To copy entire directories you'll need to add the -r parameter (recursive):
mv - moves or renames files and directories. It has similar functionality as the cp command which means that you can add multiple arguments and copy files and directories at the same time:
Examples: 
mv f3 f4 folder1/
mv f5 newfilename
mv f1 f10 f2 f6 file2 newfilename folder1/
Linux move command

Interact with files and directories content
cat - used to display files content
Linux cat command

less - used to control the listing when there is a long output. For example:
cd /
ls -R | less
using space key you control the display of pages
Linux less command

tail and head - used to extract a number of lines within a file. head is used to extract lines from the beginning of the file while tail extracts them from the end
important arguments that can be used with tail
   -n Number - displays the last Number lines
   -n +Number - displays the last lines starting with the +Number position
   -f - used to tail the output of a log file in real time
important arguments that can be used with head:
   -n Number - displays the first Number lines

wc - "word count" it's used to count the number of lines, characters or words within a file or from a command output. Arguments that can be used with the wc command include -w ( displays number of words), -l (displays number of lines), -m (displays number of characters)

file - displays information about a file. For example:
Linux file command

File search 
find - used to search directly in the File System. It's search command is accurate but, can take a long period of time if the OS is filled with data. Can be used with different arguments, some of the most important are:
   -type - f (file), d(directory), l(simlink)
   -name - searches for a file who's name contains the specified pattern
   -user - searches for the owner of the file
   -size - maximum or minimum size of a file

slocate - searches within a local encrypted database which indexes all data from your hard drive. By default, the db update is made automatically but, you can change that behavior from the config file or manually update it. If the database has indexed all info, the search speed is highly increased rather than searching using find command
database file: /var/lib/slocate/slocate.db 
configuration file: /etc/updatedb.conf

grep - "The grep command is used to search text or searches the given file for lines containing a match to the given strings or words. By default, grep displays the matching lines. Use grep to search for lines of text that match one or many regular expressions, and outputs only the matching lines. grep is considered as one of the most useful commands on Unix and other Linux operating systems." Link here

egrep - "Search for a pattern using extended regular expressions. egrep is essentially the same as running grep with the -E option." Link here

  That's about it for this article folks, hope it will serve you well in enhancing your Linux knowledge. Note that most commands written in this post support many arguments that were not described in this article so please, don't forget to use man pages or help menus to discover all features available with each command. In the following articles we will continue our Linux journey and we will learn more features supported by this Operating System. Don't forget to enjoy your day and stay tuned for the following articles!
Read More
3 Jul 2014

Linux boot loader

Leave a Comment
Linux boot loader
As you my probably know the CPU processes information that is stored in RAM. This means that upon booting up the System, you will need a software that loads the kernel from the hard drive into the RAM. The software that executes this operation is called a boot loader. Linux has two boot loaders available: GRUB and LILO. Because these tools offer extra options (selecting between multiple OS, booting into single user (similar to safe mode in Windows), etc.) they are also known as boot managers. With these in mind we can say that GRUB is both a boot loader and a boot manager for Linux.

In Linux distributions, boot managers can be installed in one of the following locations:
  • MBR - the boot manager will be located in the first sector of the hard-drive which means that it will be executed just after the BIOS initialization. With GRUB you can also create multi-boot Systems so there is no need for extra software to boot other Operating Systems like Windows.
  • First sector of root partition - boot manager will be installed in the first section of the Linux partition. Placing boot manger in this location will make it "hidden" when booting up the system. You will either need to use a third party software to locate the Linux boot loader or set the Linux partition as active (the active partition is the one hosting the Operating System and it's searched by default on boot)
In /boot/grub/menu.lst you will find the GRUB configuration file. Any changes made to this file will change the GRUB's behavior upon reboot (GRUB reads the configuration file at every reboot). The conf file actually contains a set of GRUB commands which instructs the software what behavior to adopt. You can also execute these commands from the GRUB console.
I will explain some of the most common GRUB commands:
- the timeout command specifies the number of seconds that GRUB menu is displayed to the user before the default OS is booted. 
- the default command configures what Operating System will boot by default. 
- makeactive sets a partition as active
- rootnoverify(hd0,0) tells GRUB to load the OS from the specified partition (hard disk 0, partition 0) without trying to recognize the Operating System
- chainloader +1 - this command is used to tell GRUB to execute the code located in the first section of the partition. This command is used when the boot loader of a OS is located in the first sector of a partition.
- kernel /path_to_kernel_file root=/dev/hda0 [options] - locates the Operating System kernel on the specified partition. 
- initrd /boot/file - loads a file from hard drive into RAM. The file is used to boot the kernel as is also known as initial RAM disk.
We've covered the main aspects of the GRUB boot loader. There are many things that we can further discuss about this topic but, it's premature to talk about them right now. When we will get acquitted with Linux we will see how to modify and configure the GRUB config file. Hope you've understood the basics of Linux boot manager, for any misunderstandings fell free to access my comments section. Wish you all the best and stay tuned for the following articles from IT training day.
Read More
© 2014 All Rights Reserved.
IT training day & Powered By BloggerHero