tag:blogger.com,1999:blog-14915049943285221102024-02-08T02:50:31.843-08:00Tivoli Storage ManagerMichael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-1491504994328522110.post-80201517044447316862010-12-18T07:12:00.000-08:002010-12-18T07:12:39.925-08:00Querying the Tivoli Storage Manager DatabaseWhile the Tivoli Storage Manager system gives a lot of options to gather data from the system, there are times when what you want to know is not available, even from the admin command line (you will never see me in the web-based admin interface). The TSM database can be queried to produce this other information. And that information can be dumped to text files allowing import into documents and spreadsheets.<br />
<br />
So, how do you query the TSM database? From the admin command line. That makes the first step to open the command line. It will be installed on your server and available for use. Just go to the TSM program group and select Administrative Command Line. Then enter the admin's user name (admin) and password. If the password is still admin, you really should change it!<br />
<br />
Now you are ready to start entering your query commands. The syntax is the same as in standard queries. But I will bet you are wondering what tables to query. How do you find that out? As usual, TSM has some awesome hep for this. In fact, when working with the command line interface I often keep two windows open. One for command entry and one for help. So, the easiest path now is to open a second instance of the command window and type - Help Select. You will now see in the first page of the help text how to find the tables in TSM, the columns in those tables and valid values for the columns that need that. So in the other window, enter select * from syscat.tables. TSM, of course, gives the results in a descriptive formatted style with description of each table.<br />
<br />
Now that you see what the tables are, let's try a simple query that I have used at many sites to see what is in the system. Type select count(*) from adsm.contents. You will probably get a warning that this may take a bit of time to complete. So go get a cup of coffee or lunch. Mine came back fairly quickly (time to get coffee) with a result of 1,055,163. This tells me that this is the number of files stored in my TSM system. Since it just backs up my personal network, yours will probably be a lot larger. Next you may want to see what columns are available for the adsm.contents table. You could do this by select * from syscat.columns where tabname = 'CONTENTS' (and the uppercase letters are important). Now you see all the column names for CONTENTS with the description of each one. What could be easier to help in building a query? <br />
<br />
Next I did another query that may take some time to complete. What this one will give you is the number of files in your TSM system for each node on the network. Then you can go holler at the people who are taking more than their share. select node_name, count(node_name from adsm.contents group by node_name order by node_name Again, not sure how long this will take on your system. Mine is an overworked older server so it was slow. But once it completes, the count of file usage for each node will be available.<br />
<br />
One last item and then you are free to go and see what you can do with this portion of TSM. I said you could dump this info to text files and import them into documents and spreadsheets. How to do this is total simplicity as with most of TSM. Try this command. Make sure the folder you plan on entering here does exist. Select * from syscat.columns where tabname = 'CONTENTS' > C:\TSM_Query\Output\ContentsColumns.txt And hit enter. Now go to that folder and you will see a text file with a nice formatted output from that query. This information can then be imported into an Excel spreadsheet, included in a document, or just emailed as is to someone in need of your query results.<br />
<br />
So go explore the available tables and columns, then impress your boss by creating a formatted report of data stored in TSM and do it in very short time. Or take a break in the middle. You will still impress him but will make it look harder to do.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com1tag:blogger.com,1999:blog-1491504994328522110.post-3403137401743252382010-11-29T09:38:00.000-08:002010-11-29T09:38:54.910-08:00TSM Database Page Shadow FileThe Tivoli Storage Manager database has a file calle the Page Shadow File (dbpgshdw.bdt). This file is a mirror of transactions that are written to the database. This occurs only if two options are enabled in the Server Options file (dsmserv.opt). <br />
<br />
DBPAGEShadow Yes<br />
<br />
MIRRORWrite DB <span lang="EN">Parallel</span><br />
<br />
<span lang="EN">By default TSM installs with the first set to Yes and the second set to Sequential. This causes the Page Shadow File to not be used. However, even with the Parallel option being chosen, the Yes option still causes the Tivoli system to require that file. In fact, one of the big ways I have seen a Tivoli Storage Manager database to get corrupted is having the parameters set as above and then some process lock the Shadow File. I have seen machine backups that lock each file as they are backed up do this. When that happens, you better have a good current back up of the database available.</span><br />
<br />
<span lang="EN">The Page Shadow File is implemented in TSM to help with high volume systems that may get overwhelmed in database updates. The transactions are written to this file allowing them to be picked up later and processed when there is less system activity. If you have a system with heavy activity like this, I would encourage use of the shadow file. I would make sure it is in a location where the system backup will not be touching it. There is no need for this file to be backed up, and as I said above, locking this file is dangerous. The location of this file can be determined int he dsmserv.opt. By default the TSM server looks for it in the server directory that was created when the server instance was created. By changing the entry to a fully defined path, it can be relocated to any directory on the server. </span><br />
<br />
<span lang="EN">If your system is not high volume, I would reccomend changing the default DBPAGEShadow from Yes to No. Once this is done, you can delete the file and all is fine. Or it can be left alone since the system will no longer look for it. This will keep any harm from this file and keep it from causing database corruption if the file is inadvertantly locked by another program.</span>Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com1tag:blogger.com,1999:blog-1491504994328522110.post-11205698217640009452010-11-24T09:54:00.000-08:002010-11-24T09:54:55.090-08:00Copy Storage PoolsOK, you have TSM backing up your entire enterprise. All important data from data from the workstations and servers in your business is being efficiently stored in storage pools on a large san unit. Each day people are restoring items that were deleted by accident or were needed after it was thought they were no longer needed. All is going wonderfully. Right up until a disk gets corrupted in the san. But they are mirrored? Guess what? The corruption was mirrored as well. (Seen this happen). Now you can spend days getting the affected storage pools recreated. And since the storage was spread across multiple drives, this means a lot of the storage pools are damaged.<br />
<br />
So what would copy storage pools do for this? If the copy storage pools were available, the TSM system would simply get the files it was looking for from the copies and start automatically recreating the damaged files. The copy storage pools can also be used to totally recreate any damaged storage pools. But the main thing is that while files may be restored slightly slower, the users will never actually know that anything happened. They will get their files restored and no one will be calling for support.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-6270159134173532442009-12-05T09:04:00.000-08:002009-12-05T09:33:59.667-08:00TSM with Content ManagerSorry for the complete lack of posts here lately. My business, Transcendent Technology, has been doing fantastically. That part is great as it means a good income for me. The downside is, of course, that I have been working some really long weeks. Many 14 hour days and 7 day weeks. That does not leave a lot of time to post info on here. Thought I would take a shot though and get some new info on here. No one is asking questions, so I will just talk a little of my basic background in TSM and them talk of some issues on upgrading to new versions.<br /><br />My main usage of TSM is in conjunction with Content Manager and Content Manager OnDemand (see my OnDemand blog at http://contentmanagerondemand.blogspot.com/). IBM has leveraged the TSM product to provide a storage repository for these products. I will use an old client as an example (no names to protect me!). I installed Content Manager for this client several years ago. I believe it was in the early days of version 8 (horror story of it's own). The client insisted they did not want TSM installed with the product. Against our better judgment we finally agreed. About a year later, the client had almost a TB of 40K files stored on their hard drive. The hard drive got corrupted and the corruption was written across the Raid. The entire TB was ruined. They started restoring this data from file system backups. Took them 30 days to get this finished. I found out all this as I was up there installing a new version of Content Manager, with TSM support. I was also running queries to get the current documents to move from disk to TSM.<br /><br />So why is TSM such a plus in this situation? The way that IBM has leveraged TSM is to use the API's and backup the documents from the disk drive to storage pools in TSM. The storage pools can be on any supported hardware, although most of them are on dasd. Once the document has moved to TSM it is deleted from disk storage. In fact Content Manager can be configured so that the document goes straight to TSM and never is stored on disk. If the storage pool volumes are on dasd, this is even the recommended approach when I install Content Manager.<br /><br />But the real advantage of TSM in conjunction with Content Manager or OnDemand is the idea of storing the multitude of small files that are the usual report data in these two products into large files in TSM. So, with the client I mentioned earlier, as I moved their 40K documents into TSM, I configured 20G storage pool volumes. One TB would be 50 of these volumes. Now the file system backups run quicker, having only to backup 50 files instead of millions and a restore would take much less time. SO why does it take less time to restore 50 20G files as opposed to the same amount of disk usage in small files? Because of opening and closing all the small files. There is system overhead for opening and closing the files as they are restored. Opening a single 20G file and streaming the data back into it takes a fraction of the time as would restoring 20G of 40K files. The same is true when doing system backups. The larger files backup much quicker than a bunch of small ones.<br /><br />So bottom line, if you are using either Content Manager, or Content Manager OnDemand installing TSM to store the documents and reports is a no brainer to keep the file system in a manageable state. If you are not storing enough documents in either of the products to really need TSM then you should not have these products. Go for a lower end product.<br /><br />Next post (whenever it may come) will cover the part on upgrading a system that I mentioned earlier in this one.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com1tag:blogger.com,1999:blog-1491504994328522110.post-27482943935777828152009-05-16T08:24:00.001-07:002009-05-16T08:43:16.756-07:00TSM Device Configuration FileI have seen some questions on the <span class="blsp-spelling-error" id="SPELLING_ERROR_0">internet</span> about the <span class="blsp-spelling-error" id="SPELLING_ERROR_1">Tivoli</span> Storage Manager Device Configuration file. This file is generated by the command<br /><br /><span class="blsp-spelling-error" id="SPELLING_ERROR_2">BAckup</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_3">DEVCONFig</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_4">Filenames</span>=<where><br /><br />So why backup the device configuration? Most of the time you will not need it. But what if you need to restore a database that has failed. Where is the device information usually stored? in the database. What is one of the things you need to restore the database? The device configuration so the <span class="blsp-spelling-corrected" id="SPELLING_ERROR_5">restore</span> command will know the definition of the device being used to restore it. Without the <span class="blsp-spelling-error" id="SPELLING_ERROR_6">devconfig</span> file, this becomes a real Catch-22. Can't <span class="blsp-spelling-corrected" id="SPELLING_ERROR_7">restore</span> without the devices and can;t get the devices without restoring.<br /><br />So, what does a <span class="blsp-spelling-error" id="SPELLING_ERROR_8">devconfig</span> file look like? Here is a simple example:<br /><br />/* Device Configuration */<br />DEFINE <span class="blsp-spelling-error" id="SPELLING_ERROR_9">DEVCLASS</span> BACKUPS <span class="blsp-spelling-error" id="SPELLING_ERROR_10">DEVTYPE</span>=FILE FORMAT=DRIVE <span class="blsp-spelling-error" id="SPELLING_ERROR_11">MAXCAPACITY</span>=5242880K <span class="blsp-spelling-error" id="SPELLING_ERROR_12">MOUNTLIMIT</span>=10 DIRECTORY="F:\<span class="blsp-spelling-error" id="SPELLING_ERROR_13">TSM</span>_BACKUP_FILES" SHARED=NO<br /><br />DEFINE <span class="blsp-spelling-error" id="SPELLING_ERROR_14">DEVCLASS</span> CM_COPY <span class="blsp-spelling-error" id="SPELLING_ERROR_15">DEVTYPE</span>=FILE FORMAT=DRIVE <span class="blsp-spelling-error" id="SPELLING_ERROR_16">MAXCAPACITY</span>=102400K <span class="blsp-spelling-error" id="SPELLING_ERROR_17">MOUNTLIMIT</span>=1<br />DIRECTORY="F:\<span class="blsp-spelling-error" id="SPELLING_ERROR_18">TSM</span>_BACKUP_FILES\CM_COPY" SHARED=NO<br /><br />DEFINE <span class="blsp-spelling-error" id="SPELLING_ERROR_19">DEVCLASS</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_20">FILENET</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_21">DEVTYPE</span>=SERVER <span class="blsp-spelling-error" id="SPELLING_ERROR_22">MAXCAPACITY</span>=2097152K <span class="blsp-spelling-error" id="SPELLING_ERROR_23">MOUNTLIMIT</span>=256 <span class="blsp-spelling-error" id="SPELLING_ERROR_24">MOUNTRETENTION</span>=0 PREFIX=<span class="blsp-spelling-error" id="SPELLING_ERROR_25">ADSM</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_26">SERVERNAME</span>=<span class="blsp-spelling-error" id="SPELLING_ERROR_27">FILENET</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_28">RETRYPERIOD</span>=10 <span class="blsp-spelling-error" id="SPELLING_ERROR_29">RETRYINTERVAL</span>=30<br /><br />DEFINE SERVER <span class="blsp-spelling-error" id="SPELLING_ERROR_30">FILENET</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_31">COMMMETHOD</span>=<span class="blsp-spelling-error" id="SPELLING_ERROR_32">TCPIP</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_33">HLADDRESS</span>=<span class="blsp-spelling-error" id="SPELLING_ERROR_34">filenet</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_35">LLADDRESS</span>=1500 <span class="blsp-spelling-error" id="SPELLING_ERROR_36">NODENAME</span>=<span class="blsp-spelling-error" id="SPELLING_ERROR_37">GROUCHO</span> PASSWORD=215f15799f38d12750109<span class="blsp-spelling-error" id="SPELLING_ERROR_38">df</span>0c1<span class="blsp-spelling-error" id="SPELLING_ERROR_39">fb</span>5e81f6 <br /><span class="blsp-spelling-error" id="SPELLING_ERROR_40">SERVERPASSWORD</span>=215f15799f36d12750107<span class="blsp-spelling-error" id="SPELLING_ERROR_41">df</span>0c1<span class="blsp-spelling-error" id="SPELLING_ERROR_42">fb</span>5e81f6<br /><br />SET <span class="blsp-spelling-error" id="SPELLING_ERROR_43">SERVERNAME</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_44">GROUCHO</span><br /><br />DEFINE LIBRARY <span class="blsp-spelling-error" id="SPELLING_ERROR_45">DBBACKUP</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_46">LIBTYPE</span>=FILE<br /><br />DEFINE DRIVE <span class="blsp-spelling-error" id="SPELLING_ERROR_47">DBBACKUP</span> <span class="blsp-spelling-error" id="SPELLING_ERROR_48">DBBACKUP</span>1 ONLINE=Yes<br /><br />I broke this up with carriage returns to make it a little more readable. The main line (and <span class="blsp-spelling-corrected" id="SPELLING_ERROR_49">possibly</span> the only line) that would be a necessity for a <span class="blsp-spelling-corrected" id="SPELLING_ERROR_50">restore</span> of the database is the first one. This line defines the sequential access file device class that is used for creating database backups and where to access the files for restores.<br /><br />So what does an administrator do if the database fails and this file is not available? It can be created manually and used to do the restore. Just look at the first line in this example and create a file with that type of information, based on your system. Very doable, but I would still prefer creating one at backup time. Just use the script that was laid out in the last post and you are all set.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-67853733940064181122009-03-20T16:15:00.000-07:002009-04-02T12:05:06.836-07:00Backup ScriptOK, we had been talkign about database backups before I rambled off on a couple of other topics. This post is going to talk about using the Tivoli Storage Mnager admin client scripting tool to gather all the parts of the database backup into a single scheduled set of commands.<br /><br /><br /><br />First you may ask, why the script and why not seperate schedules. The reson is that some of the steps need to occur inthe correct sequence and this makes sure that they do. For example: You do not want to take a backup of your volume history before the database backup completes or you will not have the volumes of that backup defined in it. Se here goes:<br /><br /><br /><br />Start the script with a deletion of old database backup volumes from the volume history. This keeps the volhist file from becoming too cumbersome over time.<br /><br /><br /><p><strong>def script backup "del volhist todate=today-5 type=dbb" line=001</strong></p><p>This line starts the definition of a script call backup and adds line 1 to that script. Next we will add line 2 which will do tha actual database backup.</p><p><strong>upd script backup "ba db dev=backups t=f w=y" line=002</strong></p><p>Line 2 to backup the database has now been added to the script. Note that to add an additional line we used the update rather than the define command. Since the script now exists, the rest o the lines must be added with this option. Also the backup is a full backup to allow it to be used on it's own for a restore. Finally not the w=y or wait=yes option on the backup commnad. This causes the script to wait and not process subsequent commands tillt he backup is complete. This is important as we do not want to backup the volume history until th evolues from the database backup have been created.</p><p>The next line will cover this exact item.</p><p><strong>upd script backup "backup volhistory filenames=F:\TSM_Backup_Files\volhist.out" line=003</strong></p><p>So line 3 is added to the script which creates a volume history file of all volumes in the TSM system including the ones just added by the backup. And finally line 4.</p><p><strong>upd script backup "backup devconfig filenames=F:\TSM_Backup_Files\devconfig.out" line=004<br /></strong></p>This line will create a backup of the Tivoli Storage Manager configured devices. This is needed in case the entire database is corrupted and needs to be restored. Part of the command to restore a database is the device class parameter telling the system what device to use in accessing the backup volumes defined in the volume history. Since this info is in the database, it may not be available for a database restore. That is when this file comes in. TSM uses the information in this file to recreate the device configuration before attempting to restore the database volumes.<br /><br />So now that the script is all ready, only one step remins. Putting it into the administrative schedules so that it runs on a scheduled basis to keep database backups available for a restore.<br /><br /><strong>def sch BACKUPDB t=a cmd="run backup" t=a ACTIVE=y startd=04/02/2009 startt=23:00:00 dur=1 duru=h per=1 peru=D day=ANY exp=N</strong><br /><strong></strong><br />This will schedule the backup script to run each night at 11:00 P.M. until the system breaks down or someone tells it to stop. There are quite a few parameters on this command, and most of them are default values. I just added them to ensure you knew what was going to happen to the script.<br /><br />That is enough for today. Again I am sorry I have not been adding posts as I wanted, but hope to improve in the near future. I have left the company I was affiliated with and have started to work as an independent contractor. Hopefully this works out and also leave me more time for items such as this.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com2tag:blogger.com,1999:blog-1491504994328522110.post-22888666281210014652009-03-20T15:52:00.001-07:002009-03-20T16:12:56.329-07:00Backup Client - Incremental (inc) versus Selective (sel)Both Incremental or inc and Selective or sel backups are available from the Tivoli Storage Manage backup client. These two backup methods are considerabley different.<br /><br />Incremental does mainly what it says - it backs up all files that have been added or changed on the file system since the last backup. This is affected by the exclude/include information in the configuration file. In addition, specific files can be backed up by selection from the command in the command line client. In addition the frequency and mode on the managmement class's Copy Group can affect if the file is included. The frequency affects how often the file can be backed up. The mode can cause it to be backed up whether it has changed or not.<br /><br />Selective is meant to be used for exactly what the name indicates. Selecting individual files for backup. It is to be used mainly if the backup of a file or set of files is know to be damaged. By doing a selective backup, the file can be put into the TSM server again. Care must be taken with the use of Selective backups as more files than intended can be backed up and will affect the number of inactive files being retained. If the policy is for 5 inactive copies to be retained, and a selective backup is made before the file changes then 2 of the copies are identical. <br /><br />So, unless there is a specific need, use the incremental backup rather than the selective.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-38270343091149344862009-03-13T12:40:00.000-07:002009-03-13T12:45:36.792-07:00Progressive Incremental BackupFirst, I apologize for the delay in getting a post on here. I will try to be a little quicker on updates. I have had a hard week with long hours. Haven't even been on TSM stuff. It has been all DB2/400 and Oracle related. But enough of that....<br /><br />When TSM says it is doing an incremental backup it does not do a normal incremental. Most incremental backups store any file that has changed since the last full backup. Then each incremental that follows backs up that file again. With this method, each incremental backup gets a little larger than the previous and takes a little longer to complete. As an example, take four files file1, file2, file3, file4.<br /><ul><li>Full backup - file,1, file2, file3, file4</li><li>File1 changes</li><li>First incremental - File1</li><li>File2 changes</li><li>Second incremental - File1 and File2</li><li>File3 and File4 change</li><li>Third incremental and all subsequent - File1, File2, Fle3, File4<br /></li></ul><p>So at this point you might as well be doing full backups on each pass. This will conitue until the next full backup. When Tivoli Storage Manager does a backup it goes like this: </p><ul><li>First backup - File1, File2, File3, File4</li><li>File1 changes</li><li>First incremental - File1 (original copy of file1 marked inactive)</li><li>File2 changes</li><li>Second incremental - File2 (original copy of file2 marked inactive)</li><li>File3 and File4 change</li><li>Third incremental - File3, File4 (original copies of file3 and File4 marked inactive)</li><li>File1 changes</li><li>Fourth incremental - File1 (Second copy of File1 is marked inactive)</li></ul><p><br />As you can see, the Progressive Incremental that TSM uses is much more effecient than the than a true incremental. In fact there is never a need for a full backup. You can force one, but the incrementals keep a full set of the current files on the system along with a configurable number of inactive files. </p><p><br />How does TSM keep track of what is new and what has been backed up before? Through it's database. It actually checks every file on the system during an incremental backup and only copies the ones to it's storage pool volumes that have changed. This reduces the amount of traffic on the network while keeping a complete set of the computers data available for restore in the event of loss.</p>Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com2tag:blogger.com,1999:blog-1491504994328522110.post-25257580342608035962009-03-09T12:34:00.000-07:002009-03-09T13:13:53.784-07:00TheTivoli Storage Manager Activity LogTivoli Storage Manager keeps track of all of the server's activity in a database table named actlog, or activity log. The question I was asked (which, of course, triggered this post) is whether entries in this log can be deleted. The answer is no. Individual entries cannot be removed and the log as a whole cannot be cleared. With that said, there are ways to restrict either the amount of data stored in the log or the amount of data displayed from the log.<br /><br /><br /><br /><strong>Restricting the Actlog Size:</strong><br /><br />There are two ways to restrict the size of the activity log or to shit it off entirely. The first of these is by days of retention:<br /><br /><strong>set actlogretention 1 mgmtstyle=date</strong><br /><br />This command says to keep activity log info for 1 day. The number 1 can be replaced by any value from 0 to 9999 to indicate how many days of log info to keep. If set to 0, then nothing is retained and the log is efectively shut off. This value can be changed as desired. So if something is going to be done that will generate a large number of activity records, just set it to 0 and set it back when done.<br /><br />The second way to restrict it is by size (I will use the shorthand for this example):<br /><br /><strong>s act 1 m=s</strong><br /><br />This version of the command is to set the mgmtstyle (m) to size. The example will keep the activity log from exceeding 1 M in size. Again, the valid vlaues are 1 to 9999.<br /><br /><strong></strong><br /><br /><strong>Restricting the Actlog Query:</strong><br /><br />If restricting the size is not possible then restricting the results from the query is very possible. There are two ways to query the information from the activity log. The most common is the<br /><br /><strong>query actlog (q act)</strong><br /><br /><strong></strong><br /><br />The second is to use the SQL version to do:<br /><br /><strong>Select * from adsm.actlog</strong><br /><br />(The biggest drawback of the second format is that the data is returned with 1 column per line)<br /><br /><br /><br />Both will display the same information in a different structure. And both can be filtered. The Query actlog can be given a date and time filter with the begin and end data and time or by use of the search= parameter. The Select can be filtered the same way as any SQL by designating a Where clause. This can be used if there are a large number of log records that are not really wanted in the returned data. Snad while the select command returns the dater in a bulkier format, it does a better job of filtering the data thatn the seach= of the query option since filters such as Like, In, Not In and others like these are available.<br /><br /><br /><br />One final thought on querying the TSM Activity Log is that if you will be doing a similar query filter on the log records ona regular basis, this query could be saved as a script and run as needed. For example:<br /><br /><br /><br /><strong>def script QuerySession set sqldisplaymode wide Line=001</strong><br /><br /><strong>upd script QuerySession select date_time, message from adsm.actlog where msgno in (0482, 0407) L=002</strong><br /><br /><br /><br />After entering the above lines into the command line client (dsmadmc), the query will be saved for future use and can be executed by the command<br /><br /><strong>run QuerySession</strong><br /><br />which will return all the messages in the activity log with the message numbers ANR0482W and ANR0407I. Various scripts could be saved and executed for various sub-queries in the activity log. (See one more reason I like the command line?) Scripts are very handy in this manner and a future post will be onde on their use.<br /><br /><br /><br />So, bottom line is that once activity log messages are stored, they are stored for the duration of the retention method definition (size or days) and TSM has no method in place to remove them. But the data returned from an activity log query can be controlled. (And try out the command line interface. While intimidating because there is no point and click, it is not hard to get used to. Makes people think you know what you are doing too.)Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com1tag:blogger.com,1999:blog-1491504994328522110.post-68052523689909958642009-03-08T15:40:00.001-07:002009-03-08T15:44:21.123-07:00Open Invitation for Tivoli Storage Manager QuestionsThis posting is going to serve as a placeholder for any questions you may have on Tivoli Storage Manager. Just post your question as a comment on this posting. If I can help you out, I will add a post on that comment. Be sure and leave an email address in case I need to ask for any clarification.<br /><br />The posts up to this one are jsut some items that I thought might help people out. One of the things I am often called in to do is restore damaged databases. Sometimes this is doable. Other times people have not kept good database backups and some of their data is then lost. I am doing this series to give good backup strategies and will include info on the recovery step in case a database is lost.<br /><br />If other topics are requested, we will leave the backup and recovery for a while and go look into those items.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com2tag:blogger.com,1999:blog-1491504994328522110.post-20167248650364083722009-03-07T07:43:00.000-08:002009-03-07T08:06:55.113-08:00TSM Device Configuration (Devconfig) and Volume History (Volhist)OK, after the last post, you have a backup image of your Tivoli Storage Manager database by doing a Full backup. Then, during the night the databse on your enterprise TSM systems becomes corrupted. It is 5:00 A.M. and your CIO is breathing down your neck to get it restored. So you just need to take the volumes form the database backup and restore them, correct? Well, close. In this situation, you could, but you would have to manually create a device configuration file and specify each volume of the backup in the restore command. That will slow you down and could be next to impossible. Remember the system where they were not watching the size of the backup volumes in the device class as compared to the size of the database and had over 700 volumes in each backup? Try typing all those into a custom volume history file.<br /><br />No, rather than put yourself in this position, there are two simple commands that can be scheduled to run in parallel with the backups or can be scripted to run right after the backup. These two commands will backup the device configuration and volume history tables in your TSM database to flat text files on the file system. These are (in their shorthand form):<br /><br /><strong>ba devconfig f=F:\backups\devconfig.out</strong><br /><strong>ba volhist f=F:\backups\volhist.out</strong><br /><strong></strong><br />The first line will backup the the device configuration file. Now you ask, why do you need that? The device configuration information is in the database. Won't you get it back when you restore the database? Yes you will, but you need a device to tell TSM where to get the volumes for the restore. And if the device information is in the database and you are trying to restore the database.... Noe you get the picture. So the TSM server, when started up from the command line and told to do a restore will look for a devconfig.out file. Very handy.<br /><br />The second line backs up the volume history. This is a listing of all the volumes created for sequential disk volumes by the TSM server. These volumes can include copy storage pool sequential disk volumes as wll as the databae backup volumes. But as long as you have this file, you have the info needed to restore the database.<br /><br />And of course, the only parameter this command really has other than the type of info to backup is the F or File location where you want the information stored. The location is not just the directory, but the file in that directory. And you can designate the same file each time this runs as both of the backup commands will overwrite existing files.<br /><br />So now we have the database backup, the devcnfig backup and the volhist backup. Next time - how do we schedule or scipt these commands so that a full backup of the database with the acompanying configuration files is created?Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-90606259444166043352009-03-06T18:00:00.001-08:002009-03-07T06:16:29.593-08:00Keeping Your TSM Database SafeToday's post is going to be along the lines of the new movie Watchmen. The movie looks at who is watching the watchers. that while the super-heroes are watching over the world and taking care of keeping it safe, who is watching over them? We are going to look at who is watching the data. While Tivoli Storage Manager does a great job of backing up your data and keeping it safe, what keeps TSM safe.<br /><br /><br />The answer to that is the TSM admin. It is the responsibility of the admin to make sure that the data gathered by Tivoli Storage Manager and the meta data for that data is kept safe until it is needed. Of course in a perfect world it would never be needed because nothing would get lost or destroyed. In the real world, this does not happen. So the admin for the TSM system had better be sure the backup data and the information to access that data is available when people need it. Because not only can the servers and workstations in the domain suffer lost or destroyed data, so can the TSM system.<br /><br /><br />There are many different portions of the TSM system that need to be protected and can be very easily. The storage pools can be backed up to copy storage pools to create offsite copies of the domains data, the database information can be mirrored to multiple disk locations to lower the risk od database loss, and, as we are going to talk about today, the database can be backed up to flat files. These files can be stored offsite to keep your meta data about the other files stored in TSM as safe as possible.<br /><br /><br />The first step toward keeping the Tivoli Storage Manager database safe is to create regular Full backups and periodic Incremental backups. This type of process can be scheduled inside of TSM so that it occurs at regularly scheduled intervals. To get started we will need to create a disk device class that defines where the backup files will be stored and how large each file in the bakup set. This device class should poit to a disk separate from the disk where the TSM database volumes are stored. Putting your backups to the same disk as the database itself resides on is a very bad idea. Do this and lose a disk and you lose both the database and it's backups. So keep these separate is the first step in safety. To create the device class for this process, execute the following in the command line interface. (Remember my preference for command line from the previous post?)<br /><br /><br /><strong>define devclass backupdevclass devtype=file maxcapacity=1G directory=f:\BackupFiles</strong><br /><br /><strong></strong><br />or it can be abbreviated as<br /><br /><strong>def dev backupdevclass devt=file maxcap=1G dir=f:\BackupFiles</strong><br /><strong></strong><br /><br />The abbreviated verbage can be determined in the help text by typing just the characters that are capitolized. As we go through this, we will be referring to more of the abbreviated versions of the commands.<br /><br /><br />So with the command entered from above, it is now possible to execute the command to backup the database itself. But before doing that, another reference back to the command above:<br /><br /><br />maxcap - The value I entered of 1G should be altered to go along with the size of your database. If you datagase is less than 2 or 3 G in size, this value is fine. If it is larger, then this value should be larger. Basically, divide the size of all your database volumes by this vlaue and keep the number ov volumes to 10 or less. Other wise the volumes for a restore will get out of control. I have one I am working on where they had this volume WAY two small so that each backup of the database is creating over 700 backup volumes. Not real good for a database restore.<br /><br /><br />So, we now have the device class created with the directory to a location away from the database and the capacity set to a reasonable level. Next step is the actual TSM backup command:<br /><br /><strong>backup db devclass=backupdevclass type=full wait=yes</strong><br />or<br /><br /><strong>ba db dev=backupdevclass t=f w=y</strong><br /><br /><strong></strong><br />So, what does this do? It backups up the database to the directory defined in the file device class backupdevclass doing a full backup and it will lock the command line until it is complete causing us to wait. <strong>NOTE: All parameters that require a value with an =, never put spaces around the = sign.</strong><br /><br /><strong></strong><br />Next step is to execute this at the command line, wait for it to complete, and we have a backup of the database that can then be restored to the system. Nothing to it!<br /><br /><br />Just to mention a couple of the other parameters before we wait for the next post. If we had used Wait=No then the backup would have executed as a separate process allowing use of the command line window for other things. The status of the backup could be traced with the query process (q proc) command and the results in the query activity (q act) commands. The type of backup can not only be full, but also incremental or snapshot. I have personally never used a snapshot bakcup but do use the incremental to go with a full. I will run a full at night and incremental during the day. Of course the full is needed along with the incremtnals to get a complete inage of the database restored.<br /><br /><br />Next posts:<br /><br /><br />What other files are needed with the backup to do an easy db restore?<br />How to set up the schedule(s) to automate the db backup?<br />How do you do a restore?Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-37671112276789426372009-03-03T18:54:00.000-08:002009-03-03T19:57:31.078-08:00Admin Client Options<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.mike-julie.com/admin.jpg"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 347px; height: 350px;" src="http://www.mike-julie.com/admin.jpg" alt="" border="0" /></a><br /><br />Before I dive into the backup and restore I did this last weekend I want to give some idea of my preferred working interface. I started working with Tivoli Storage Manager administration through the WebClient that was packaged with version 5.2 and older. I always found this interface easy to work with although also a little cumbersome and out of date (although calling it out of date will seem odd for me in a moment). Then they decided to come out with the new and improved Integrated Solutions Console. And I had thought the WebClient was clumsy. The more I used the ISC the less I liked it. Everything in it seemed to be hard to locate and it took more resources from the machine than I liked. Of course, you can always add the WebClient back in to the new versions, but some of the commands don't work since the parameteres for the commands have changed.<br /><br />So with both of the available GUI clients hard to use or not truly supported I moved to the only option left - the command line client. This quickly became my admin interface of choice. It is actually much easier to use and easier to work with than wither of the others. It just takes a little time to get used to it. The biggest question that comes up is the format of the commands to enter. This is easily resolved with the great help text that come with the client installation. What I usually do, especially if it is something I am not real familiar with, is open two command prompts. One I use to enter commands while using the other to display help information and the syntax of the command (see picture above).<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.mike-julie.com/dsmopt.jpg"><img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer; width: 238px; height: 150px;" src="http://www.mike-julie.com/dsmopt.jpg" alt="" border="0" /></a><br /><br />Of course before you can do this, you need two things. The first is a dsm.opt file that leads to the server. An image of this file with the basic options in it is shown in the second image. The opt file is displaying the three basic options needed to connect:<br /><br /><span style="font-weight: bold;">commmethod TCPIP</span> - How is the client going to communicate with the server. This is the default method and perfectly fine for this method of communication. NamedPipes is a good option when there is close communication and a lot of traffic, but this is best for what we are doing.<br /><br /><span style="font-weight: bold;">tcpport 1500</span> - If it is going to communicate on tcpip it needs a port. This is the standard default.<br /><br /><span style="font-weight: bold;">tcpserveraddress localhost</span> - Name or IP address of the server. If you are working from the server as I am then localhost is fine.<br /><br />This is enough to get the admin client connected. But what are you connecting. This is the second of the two items I mentioned you would need. There are two ways to start the admin. Usually when the client is installed, a link is created off the start button for the Administrative Command Line. Another option is to navigate to the baclient under the TSM folder. The name of the executable is dsmadmc. By opening a Windows command line and navigating to this location, you can enter dsmadmc and hit enter. The advantage to doing it this way is that if you mistype the password or exit the client for any other reason, just retype dsmadmc and try again.<br /><br />Once the dsmadmc is started you will be prompted ro first a user name and then a password. Once those are entered you are ready to go. When you are done, just type quit and the command line will close up.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0tag:blogger.com,1999:blog-1491504994328522110.post-31292669771868833212009-03-02T11:25:00.000-08:002009-03-02T11:48:13.908-08:00IntroTivoli Storage Manager, or TSM, is in my opinion one fo the finest pieces of sodtware IBM sells. Very reliable, very flexible, very easy to use. It is one of those items that is simple to alter the configuration of on the fly and will keep running for years with only a little simple maintanence.<br /><br />A little of my background. I have worked with Tivoli Storage Manager for the last 10 years in my position as a Systems Engineer for an IBM business partner. I have used it predominately as a storage repository for the Content Manager suite of document manager software products. In this poition I have had expierence deploying maintaining the TSM Server and the TSM Backup client. I have had expierence moving Tivoli servers from one machine to another, setting up server to server communication for data redundancy, and many other aspect's of TSM. I enjoy working with it and have worked past a lot of the quirks in the product (or more correctly the product documentation) over the years.<br /><br />On this blog I plan on sharing some of those experiences and answering any question that may arise about Tivoli Storage Manager, the TSM Server or the TSM CLient. As an example, I just finsihed this weekend relocating the TSM server form an aging server to a brand new machine. It took about 4 hours to back up the database, copy it to the new server and restore it. In doing a test run of this backup and restore, I ran into a few of the gotchas that I remembered hitting in the past and had to try and remember how to handle them on this time around. Obviously I did it as the client is up and running very well this morning.<br /><br />Tomorrow I will detail how that proces was done, step-by-step, and include th eitems that were either potential or real slow downs. Then in future days I will go over the things I am doing on the server to make it ready to work for the client for the next several years.<br /><br />If you see things in any of my posts that raise questions, please be sure to ask. I want this to be an interchange between myself and you readers. Not just a lecture from me on what I have done.Michael Ballardhttp://www.blogger.com/profile/00848496225352736921noreply@blogger.com0