Some helpful Exchange commands to know

I used to think that I could be considered a ‘seasoned vet’ when it came to Exchange. But that was back before the 2007 release… The new Exchange Management Shell has definitely taken some getting used to. Very powerful – yes. However, you’re still stuck in this small Windows ‘cmd’ screen with it’s poor formatting and unhelpful errors.

Slowly but surely, I’m getting used to the new interface and tools. Here are three commands that I use on a regular basis for reporting and status updates.

This command prints out a sorted list of mailbox sizes and users

Get-MailboxStatistics | Sort-Object TotalItemSize -Descending | ft DisplayName,@{label="TotalItemSize(KB)";expression={$_.TotalItemSize.Value.ToKB()}},ItemCount

 

Next, let’s show the current status of replication for a mailbox store. MAIL01 is our master and MAIL02 is being replicated to.

Get-StorageGroupCopyStatus -Identity MAIL01\StorageGroup1 -StandbyMachine MAIL02 | fl

 

The last one is very important for backups. It checks to see if the Exchange backups are properly working. The file you need to check is the .chk file in the storage folder.

# eseutil /mk M:\StorageGroup2\e03.ck
Output should look like this:
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server Version 08.03 Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Checkpoint file: M:\StorageGroup2\e03.chk
LastFullBackupCheckpoint: (0x0,0,0)
Checkpoint: (0xCDCD,300,0)
FullBackup: (0xCBED,8,16)
FullBackup time: 09/28/2011 14:17:45
IncBackup: (0xCD2B,8,16)
IncBackup time: 09/29/2011 20:15:15
Signature: Create time:03/07/2011 15:19:46 Rand:852206513 Computer:
Env (CircLog,Session,Opentbl,VerPage,Cursors,LogBufs,LogFile,Buffers)
( off, 652, 32600, 16384, 32600, 2048, 2048,2000000000)
Operation completed successfully in 0.63 seconds.

Microsoft Office Communicator 2007 Got’cha

 We have an Office Communicator 2007 setup in my office. I’ve found this communication tool beneficial for our office environment. Two of the biggest benefits from this application is that people’s current status is sync’d with their Outlook Calendar. So if you have a meeting at 1PM, your status icon will immediately change to Red and mark them as "busy". This status indicator shows up in several places, including the IM contact list, anywhere in Outlook that their address is defined (address book, emails, tasks, etc.), and Sharepoint.

The second huge benefit that I’ve found in Communicator is that all conversations are saved and organized in a folder within my Outlook Mailbox. There are many more benefits that come with this program but these are my two favorite.

We recently ran into an issue with one someone’s computer not performing any of these tasks. Communicator worked just fine but these two "auto" features were not working and no errors were being reported. After quite a bit of troubleshooting and searching, I finally found an explanation on Microsoft’s website. This particular user had Office 2010 64-bit installed for the additional benefits in Excel. As it turns out, Office x64 is not at all compatible with Office Communicator due to this and there is no work-around.

Communicator integration    If you install the 64-bit version of Office, certain Microsoft Office Communicator 2007 features that involve integration with Outlook are lost. For example, you cannot right-click a Communicator contact to schedule a meeting, send an e-mail message, or find previous conversations. This is because Office Communicator R2 is 32-bit, and Messaging Application Programming Interface (MAPI) calls do not cross the 32-/64-bit boundary.

More Info can be found here: office.microsoft.com/en-us/word-help/choose-the-32-bit-or-64-bit-version-of-microsoft-office-HA010369476.aspx

No word yet on if their newly re-branded product, Microsoft Lync fixes these issues or not. 
Hopefully this can save someone else a few hours of frustration getting these two to play nice.

 

The magic of the screen command

The Linux command prompt without ‘screen’ is like using a browser without tabs. I used to have 10-15 different server sessions open in different putty windows. This was a nightmare to alt+tab through to find the right server. This was also opened the door for accidentally executing a command on the wrong server. Let me welcome you to the magic of the screen command. With this command you can have limitless sessions open all in a single putty window. These sessions can have whatever you want in it and are completely separate of each other – same server, multiple servers, your game of snake, etc. The first thing to get an effective screen session setup is to setup your .screenrc file. Mine is setup with colors, 30,000 lines of scrollback, and 7 different server lables on the bottom. Note that screen starts its session numbers at 0: .screenrc:

termcapinfo xterm [email protected]:[email protected]
defscrollback 30000
vbell_msg "[[[ ding ]]]"
vbell off

hardstatus alwayslastline
hardstatus string '%{= kG}[ %{G}%H %{g}][%= %{=kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%Y-%m-%d %{W}%c %{g}]'

screen -t util01 0
screen -t svr01 1
screen -t svr02 2
screen -t svr03 3
screen -t svr04 4
screen -t svr05 5
screen -t svr06 6

Now that our screenrc file is set, just type the command ‘screen’ to start up a new screen session. This session will start with 7 "tabs" for each of the servers that were defined in the screenrc file. You will need to go into each "tab" and ssh to that server. Here are some helpful commands once in screen. Ctrl+a is how you start sending pretty much any command to screen. So ctrl+a then another character. This list is huge (‘man screen’ for a full list) but these are the ones that I find most helpful:

[space]     switch to the next window in line
#     switch to the specified screen number. If you have screens above #9 then you'll need to issue an apostrophe followed by the desired screen number
A     a capital A will allow you to rename the current screen's title
d     detach from the current screen session
K     Kill the current window. Very helpful if the window freezes for some reason

In addition to these, the ‘screen -ls’ bash command will show you screens that are currently attached to a socket. So if your connection to the server where you had your screen session open dies, all you need to do is reconnnect to that server and then reconnect your screen session. Your screen session does not die when you disconnect from the server. For example, I just left work and VPN’d into my network at work. I then opened a putty session to the server where my screen sessions live.

[[email protected] ~]$ screen -ls
There is a screen on:
        4161.pts-0.util01v      (Detached)
1 Socket in /var/run/screen/S-jchapman.

[[email protected] ~]$

I see that the screen session is already detached. This means that I can reattach it to my current session. If it wasn’t detached, then you would have to first detach it with -d and then reattach it to the current session with -r. So this command will reattach the screen session that is listed above:

[[email protected] ~]$ screen -d 4161.pts-0.util01v

If there’s only one session listed (as there is in this example) I can also leave out the screen session name parameter and it will just connect to that one You can do all sorts of cool things like screen sharing. Hopefully this is enough for you to get up and running with screen. The man page is a great source for more information as well.  

PS1 – Changing the default bash prompt

There are all sorts of nifty tricks to customizing your bash prompt. I’m often switching between so many different servers that it its essential for me to always know the current user, server name, and current server time.

For customizing this, we have the PS1, PS2, PS3, PS4 session variables. I’m only going to touch on PS1 since that’s all I need to show me the current hostname and timestamp.

From the bash man pages:

       PROMPT_COMMAND
              If set, the value is executed as a command prior to issuing each primary prompt.
       PS1    The  value  of this parameter is expanded (see PROMPTING below) and used as the primary prompt string.  The default value is
              ââ\s-\v\$ ââ.
       PS2    The value of this parameter is expanded as with PS1 and used as the secondary prompt string.  The default is ââ> ââ.
       PS3    The value of this parameter is used as the prompt for the select command (see SHELL GRAMMAR above).
       PS4    The value of this parameter is expanded as with PS1 and the value is printed before each command  bash  displays  during  an
              execution  trace.   The  first  character  of PS4 is replicated multiple times, as necessary, to indicate multiple levels of
              indirection.  The default is ââ+ ââ.

First, you can view the current variable by echoing it:

[[email protected] ~]#echo $PS1
[\[email protected]\h \W]\$
This is the default on all of my CentOS/RHEL 5.x servers. 
The bash man pages show us a full list of options and their meanings:

PROMPTING
       When executing interactively, bash displays the primary prompt PS1 when it is ready to read a command, and the secondary prompt PS2
       when  it  needs more input to complete a command.  Bash allows these prompt strings to be customized by inserting a number of back-
       slash-escaped special characters that are decoded as follows:
              \a     an ASCII bell character (07)
              \d     the date in "Weekday Month Date" format (e.g., "Tue May 26")
              \D{format}
                     the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results  in  a
                     locale-specific time representation.  The braces are required
              \e     an ASCII escape character (033)
              \h     the hostname up to the first â.â
              \H     the hostname
              \j     the number of jobs currently managed by the shell
              \l     the basename of the shellâs terminal device name
              \n     newline
              \r     carriage return
              \s     the name of the shell, the basename of $0 (the portion following the final slash)
              \t     the current time in 24-hour HH:MM:SS format
              \T     the current time in 12-hour HH:MM:SS format
              \@     the current time in 12-hour am/pm format
              \A     the current time in 24-hour HH:MM format
              \u     the username of the current user
              \v     the version of bash (e.g., 2.00)
              \V     the release of bash, version + patch level (e.g., 2.00.0)
              \w     the current working directory, with $HOME abbreviated with a tilde
              \W     the basename of the current working directory, with $HOME abbreviated with a tilde
              \!     the history number of this command
              \#     the command number of this command
              \$     if the effective UID is 0, a #, otherwise a $
              \nnn   the character corresponding to the octal number nnn
              \\     a backslash
              \[     begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
              \]     end a sequence of non-printing characters

       The command number and the history number are usually different: the history number of a command is its  position  in  the  history
       list,  which  may  include commands restored from the history file (see HISTORY below), while the command number is the position in
       the sequence of commands executed during the current shell session.  After the string is decoded,  it  is  expanded  via  parameter
       expansion,  command substitution, arithmetic expansion, and quote removal, subject to the value of the promptvars shell option (see
       the description of the shopt command under SHELL BUILTIN COMMANDS below).

 

By looking at this, we can string together what we’re looking for:

[\t \[email protected]\h \W]$

This will show us the current time in seconds, the username, the hostname, and then the current working directory. For example:

[16:25:36 [email protected] /]$

To test it out, you can simply set your current bash variable:

[[email protected] ~]# PS1="[\t \[email protected]\h \W]$ "

If you like what you see, you can make it permanent by adding it to your .bashrc file (/home/user/.bashrc). This is assuming that you only want to modify your profile. otherwise you’re going to want to look in /etc/bashrc

Hard lesson from Dell’s PERC H700 Battery Write Cache

We ran into a huge issue recently… At about 9PM on a Friday night our primary MySQL database began slowing to a crawl. This is an extremely slow time of day for our web application so we were all quite confused. All of our tables live on a pretty beefy SAN and everything checked out clear there.

Lone behold, the Battery on our Dell Controller (PERC H700 Integrated) decided that it was time to re-learn its battery cycle.

/var/log/messsages

 Sep 16 17:36:31 DB01 Server Administrator: Storage Service EventID: 2176 The controller battery Learn cycle has started.: Battery 0 Controller 0
 Sep 16 17:37:36 DB01 Server Administrator: Storage Service EventID: 2415 Controller battery is discharging: Battery 0 Controller 0
 Sep 16 17:37:36 DB01 Server Administrator: Storage Service EventID: 2248 The controller battery is executing a Learn cycle.: Battery 0 Controller 0
 Sep 16 18:37:52 DB01 Server Administrator: Storage Service EventID: 2278 The controller battery charge level is below a normal threshold.: Battery 0 Controller 0
 Sep 16 18:37:52 DB01 Server Administrator: Storage Service EventID: 2188 The controller write policy has been changed to Write Through.: Battery 0 Controller 0
 Sep 16 18:37:53 DB01 Server Administrator: Storage Service EventID: 2199 The virtual disk cache policy has changed.: Virtual Disk 0 (Virtual Disk 0) Controller 0 (PERC H700 Integrated)

It turns out that this re-learning task happens by default on Dell servers every 90 days. While our Data didn’t reside on local disk, our binlogs did. This was apparently enough to bring MySQL to a crawl.

The only way to change the behavior permanently is in the BIOS of the controller, there we can set it to only warn us that it needs to be checked. We can use some of Dell’s Open Manage tools to get more information on the status of the battery.

The first command here shows battery’s status (with example output)

[[email protected] /]$ omreport storage battery controller=0 battery=0
Battery 0 on Controller PERC H700 Integrated (Embedded)

Controller PERC H700 Integrated (Slot Embedded)
ID                        : 0
Status                    : Non-Critical
Name                      : Battery 0
State                     : Degraded
Recharge Count            : Not Applicable
Max Recharge Count        : Not Applicable
Predicted Capacity Status : Ready
Learn State               : Due
Next Learn Time           : 13 days 2 hours
Maximum Learn Delay       : 7 days 0 hours
Learn Mode                : Auto
[[email protected] /]$

We’re unable to disable the learn cycle all together. However, we can push out when it happens to 7 days from now with this command which adds 7 days to end end of the learn cycle. This cycle should still be run, just preferably at a very off-peak time when we can monitor it

 omconfig storage battery action=delaylearn controller=0 battery=0 days=7

 

The last command here forces write back cache even if battery is not available.

omconfig storage vdisk action=changepolicy writepolicy=fwb controller=0 vdisk=0

If you’re not much of a command guru, using Dell’s Open Manage GUI can also show you the status.
If you see this first image, a battery learn cycle is about to begin

 

If you see this second image, a battery learn cycle is currently in progress! You may see some of the symptoms that I mentioned above

 

For some more info on this issue, I found a nice blog posting here: http://yo61.com/dell-drac-bbu-auto-learn-tests-kill-disk-performance.html

Hello world!

Hello world! Welcome to my first post.
If you aren’t familiar with me, my name is Joe and I’m a seasoned systems engineer with specialties in a variety of areas. These include Linux, Windows, MySQL, Apache, PHP (and programming), MS Products, and so forth. If you really want a full scope of my expertise, check out my Resume.
This blog isn’t about me but rather the things I learn (tips+tricks), find, or just think are really cool in the SysAdmin world. It seems that I come across something new and exciting nearly every day, so this is where I’m going to begin posting that.
I’m just about to start a new job which includes a 30 minute commute (I’m not driving) where I expect to have time to update and discuss.
I hope you enjoy and learn something new along with me.
-Joe