SMART Cook Book

Reducing IRS Hi-Res Staring Observations

Usage

This recipe assumes that you have IRS high resolution data (From either of short high or long high modules) taken in staring mode. (IE. Single point observations.)
It further assumes that there were multiple exposures taken in each of the two nod positions.
The result will be a full end-to-end high resolution spectra output into a text or ".fits" file that can be imported into a different plotting or text processing program for publication.

|Note : |SMART has a large number of buttons, reflecting its multiple functionality. This recipe only specifically refers to those buttons actually used for this particular processing.
Any buttons not referred to do not need to be used here and may be safely ignored. |

To extract an IRS Hi-Res Staring Observation do the following :

1. SMART must be installed or accessible on the machine you are using. This machine must have a windowing system (eg. X-windows) running.
   (Check with your system administrator if you need either SMART and/or windowing installed.)
2. Invoke SMART. This can be by typing "smart" or "smart-cvs" at the prompt.
   (Check with your system administrator which configuration of SMART you have installed and how you need to invoke it.)
   Some messages will scroll past the prompt. This is normal. Then the Project Manager window will appear, headed "SMART Project".
3. All work in SMART is organized into "Projects". You can start a new project (step 4) or add to an existing one (step 5).
4. To start a new project :
   
   
   1. Click on "Project" and then select "New" on the drop-down menu that appears.
   2. A box appears called "New Project".
   3. At "New Project Name:" type in a new project name (eg. "Project1") and click "OK".
   4. The new project window will disappear and a new project will be created in SMART.
   5. An entry "Dataset1" will appear in the "Project Manager" box.
5. To add to an existing project :
   
   
   1. Click on "Project" and then select "Load" on the drop-down menu that appears.
   2. A selection window will appear headed "Please Select a File".
   3. You can navigate between different directories using the "Directories" and "Filter" boxes.
   4. To select a file click on it. It will highlight. All SMART projects have a ".smp" extension.
   5. Then click "OK" on the selection window.
   6. The selection window will disappear and the selected project will be loaded into SMART.
      

      |Note : |Depending on the project size SMART may take 1-2 minutes to load it. Messages will scroll at the prompt during loading. This is normal. |

   7. The Project Manager will display a list of current datasets in the project. Click on the one you want to add data to and it will highlight.
   8. Alternatively you can add a new, empty dataset by clicking on the "Add Dataset" button at the bottom of the Project Manager.
      A pop-up window will appear headed "New Dataset". Type in a name (eg. "Dataset42") and click "OK".
      

      Note :

      Within SMART you can only work on one dataset at a time.

6. Click on the dataset you want to work with and it will highlight.
   If this is the first time since starting SMART that a dataset has been highlighted then another row of buttons will appear at the bottom of the window.
7. Click the "Add Records/Edit Dataset" button. A new window will appear headed "Dataset:Dataset1".
8. On "Dataset:Dataset1" click "Browse". A selection window will appear headed "Please Select a File".
   

   |Note : |File naming conventions and locations for SIRTF/Spitzer data are not controlled or explained by SMART. SMART can accommodate any file naming convention applied.
   Users may obtain data files directly from SSC, from the SIRTF/Spitzer Public Archive of from their own site specific data storage.
   Please see your own site or system administrator for help with this. |

9. Select the data file(s) for processing. You can navigate between different directories using the "Directories" and "Filter" boxes.
   To select a single file click on it. It will highlight. Then click "OK" on the selection window.
   Multiple entries may be selected from the list in several different ways :
   
   
   1. Continuous block - click on the first required entry to highlight it and then drag the cursor arrow downwards
      with the mouse button still pressed until the last required entry is reached. The entire block will highlight. Then click "OK" on the selection window.
   2. Continuous block (alternative) - click on the first required entry to highlight it and then scroll down with the "down arrow" key while the "Shift" button is pressed.
      Keep scrolling down until the last required entry is reached. The entire block will highlight. Then click "OK" on the selection window.
   3. Continuous block (alternative) - click on the first required entry to highlight it then move the mouse pointer down and click on last required entry
      while the "Shift" button is pressed. This will select whole block and may be faster than using the down arrow in some cases. Then click "OK" on the selection window.
   4. Discontinuous block - First use one of the above methods to highlight the entire range from first to last.
      Then de-select unwanted entries by moving the cursor to them and clicking the mouse with the "Ctrl" (control) key pressed.
      Multiple entries within a block selection may be de-selected by either :
      1. Multiple individual clicks with the mouse button and "Ctrl" still pressed. (Useful for removing scattered individual entries.)
      2. Dragging the cursor arrow downwards with the mouse button and "Ctrl" still pressed until the last required entry is reached. (Useful for removing blocks.)
      Then click "OK" on the selection window.
   5. Discontinuous individual entries - Select one by one by clicking with the mouse with the "Ctrl" (control) key pressed. Then click "OK" on the selection window.
10. The selected file(s) will appear listed in the "Dataset:Dataset1" window.
11. If necessary you can change any file name (within SMART) by clicking on it to highlight it. Then click the "Rename" button at the bottom of the "Dataset:Dataset1" window.
    A small window headed "Rename" appears. Enter the new file name in the "Rename" window and then click "OK".
    This will return you to the "Dataset:Dataset1" window with the file renamed
.
12. Click on the (first) file to be extracted. It will highlight.
13. Click on the "Extract" button. (Bottom row of buttons on the "Dataset:Dataset1" window.)
14. A new window appears headed "Extraction ...".
15. Under "Select Extraction Type:" click on "Full/!NoSkySub". (This is the default.) Then click "Exit".
    

    |Note : | "Full/!NoSkySub" is the only extraction type currently available for high resolution spectra.
    It produces the spectrum arising from all flux contained in the full five pixel height (perpendicular to dispersion) of the high resolution slit.
    There is no background subtraction with this option.
    If there is a significant background underlying the source, other methods have to be used to subtract this background, such as having an independent "sky" spectrum for subtraction. |

16. Messages will scroll at the prompt. This is normal.
17. A large window will appear titled "SMART-ISAP:ISO/SIRTF Spectral Analysis Package". This is the IDEA Window.
    At its center is the display box where extracted data is displayed and can be worked on.
18. Above the display box are a group of radio button lists titled "BCD:", "Order:", "Slit Pos:" and "Module".
    For the "Order:" list click on "All".
19. Then click the "Plot" button on the left hand side of the IDEA Window. The extracted data will appear in the display box as a set of data points.
20. Click on the "Style" button on the left hand side of the display box. A new window will appear called "Plot Style Preferences". In the top line of this ("Line Style:") click on "Connected".
    Then click on the "Apply" button at the bottom left hand corner. The "Plot Style Preferences" window will disappear.
21. WARNING : If you decide not to use or leave a particular window, (here "Plot Style Preferences"), always leave by clicking on its "Cancel" or "Quit" button
    and NOT by clicking on the "X" in the upper right hand corner. Clicking the "X" can cause unpredictable results.
22. OPTIONAL STEP - color Some users find it helpful to have the different orders displayed in different colors. To do this click on the "Style" button on the left hand side of the display box.
    A new window will appear called "Plot Style Preferences". In the second line of this ("Color Coding Preference:") click on "Order Number".
    Then click on the "Apply" button at the bottom left hand corner. The "Plot Style Preferences" window will disappear.
23. Extracted data now appears in the IDEA Window display box as connected data points.
    (Each order will also be colored differently if you have selected this above.)
    

    Note :

    If your data suddenly has one order missing it may be that one of your default colors is black. See your system administrator to change this.

24. The full spectrum can appear very ragged due to poor calibrations at the edges of each order.
    Any part of the spectrum can be enlarged for examination by holding down the left mouse button and drawing a box around the area for examination.
    When finished click "Plot" again to return to the full spectrum.
25. It is often helpful to trim the edges of each order as the first and last few data points of each order may be poorly calibrated and misleading.
    To do this first go to the "Orders" list on the IDEA Window above the main display box. Unselect "All" and select the first order only, order 11.
    Then click "Plot" on the left hand side of the IDEA Window.
26. This will display just order 11. If there is significant rise or drop at either/both edges of the order it can be masked out.
27. With the right mouse button draw a line box around the portion of the edge of the spectrum you want to mask out. A window headed "Applications" will appear. Click on "Mask".
    The "Applications" window will disappear and in the main display box the selected portion of the order will be masked out. Repeat for the other edge of the order if needed.
    

    |Note : |The data may be automatically re-scaled to best fill the display box. IE. Its appearance may change. This is normal. |

28. If you do not like a particular box you have selected click on "Cancel" at the right hand side of the "Applications" window (which will disappear) then click "Replot" at the bottom right of the IDEA Window. This will remove the box you have drawn and you can draw another.
29. At any time you can recover data points that have been masked. To do this on the IDEA Window click "All" on the "Orders" list, then "Plot". This ensures that all the data is displayed.
    Then with the right mouse key draw a box around the entire display box area within the displayed axes.
    When the "Applications" window appears click "!UnMask". This will restore the masked data to the display box.
30. If you are absolutely sure you will NEVER need the edge data points then instead of selecting "Mask" on the "Applications" window select "Zap" instead.
    The "Zap" function works in the same way as "Mask" however any data removed with it is gone permanently from the extracted data.
    It cannot be recovered or re-viewed later.
31. For each order, (orders 11-20), repeat steps 25-30 until all unwanted order edges have been removed.
32. At this point the IDEA window contains only one dataset. To check this click "Store" on the left had side of the IDEA window.
    A small window will appear headed "Stored Data Sets", which lists the datasets in use. Only one will be listed.
    To apply the "Choose and Apply Function" to a dataset on this list click on the radio button next to the dataset name and then click on "Apply Function to : [filename]" button.
    This will normally be used when other datasets are added, as explained below.
33. To co-add the extracted data with the corresponding nod position data first click on the relevant corresponding nod position file in the "Dataset:Dataset1" window.
    Do NOT quit the IDEA window to return to the "Dataset:Dataset1" window or you will lose the data already extracted and stored.
    

    |Note : |In this case it is currently considered more accurate to extract two spectra first and then co-add them.
    Co-adding before extraction may introduce errors as it cannot be certain that two different images fall on exactly the same pixels in a given array.
    Here co-adding is described in steps 44-59. |

34. Each dataset can contain several different exposure ID's and DCE numbers. If you need reminders as to which exposures are at the same nod positions on the slit,
    click on the relevant file in the "Dataset:Dataset1" window to highlight it and then click on the "View" button below it.
    You can then select one of SMTV/ISAP, IDP3/ISAP or Quickview/ISAP to see an image of the spectrum.
    This will show you which images are in different nod positions.
35. For this second file then repeat steps 12-16.
    A small pop-up window headed "Acknowledge" will appear announcing "Updating stored data sets with newly extracted data".
    Click "OK". Then repeat steps 18-21.
36. Multiple datasets can be held within the IDEA window. However only one can be worked with at any one time. The dataset being worked with is called the "Prime" dataset.
    The first dataset to be extracted is automatically made the prime dataset. Running a second or subsequent extraction does not change this.
    The first extracted dataset remains the prime dataset unless it is specifically changed. For this application it is usually fine to leave the first extracted dataset as the prime one.
    Only make another dataset prime if there is a particular need to do so. To change the prime dataset do the following :
    
    
    1. Click on "Store". The "Stored Data Sets" window will appear with a list of all datasets.
    2. Click on the one you want to make prime. (IE. work on at that time.)
    3. Click on the "Apply Function to : [filename]".
    4. On the sub menu which appears click "Make Prime". The selected dataset will then become the prime one.
    5. This may cause the display box to go blank. The new prime dataset may well have some different configuration of orders from its predecessor.
       If this happens click "All" on the "Orders" list near the top of the IDEA window, then "Plot" on the left hand side.
       This will display the new prime dataset in the display window.
37. On the IDEA Window left hand side click the "Store" button.
38. The "Stored Data Sets" window will display. This has a radio buttoned list of all currently extracted datasets, in order of extraction.
    You can now compare or combine the different stored datasets.
39. On the "Stored Data Sets" window the two datasets may have the same names.
    To prevent confusion one or both can be renamed. To rename a dataset :
    
    
    1. Select a dataset on "Stored Data Sets" by clicking on the radio button next to it.
    2. Click on the "Apply Function to : [filename] button.
    3. On the sub-menu that appears select "Change Name".
    4. A small pop-up window appears. In the "New Title:" field type the new name you want to give the dataset.
    5. Click "OK". Both the pop-up and the "Stored Data Sets" window will disappear.
    6. Click "Store" again. On the "Stored Data Sets" window the selected dataset will be listed by its new name.
40. Click "Store" to display a list of datasets. Select the second. (This recipe assumes that you have only two datasets at this point.)
    Then click on "Apply Function to :[filename]" at the top of the "Stored Data Sets" window.
    A sub-menu will appear, click on "Make Prime Data Set". The "Stored Data Sets" window will then disappear.
41. On the main IDEA Window click "All" on the "Orders" list, then "Plot". This will display the second extracted file in the main display box.
42. If you need to trim the edges of each order from this second dataset repeat steps 25-30 as necessary.
43. Once any unwanted edge effects have been removed the two datasets can be co-added.
44. Click on "Store" on the IDEA Window to display the "Stored Data Sets" window. Select the first list entry.
    Then click on "Apply Function to :[filename]". A small sub-menu will appear. Click on "Use as Reference Data Set".
    (The display will not change. Selecting this option does not change the prime dataset or the display window.)
45. With the right-hand mouse key draw a box around the displayed data.
46. The "Applications" window will appear. This allows various ways of comparing or combining the two datasets.
47. OPTIONAL STEP - Comparison If you wish a comparison of the two spectra, click on the button "Oplot data to ref".
    A window appears headed "Oplot (plot prime with reference)". At the top this window lists the two datasets as "Primary" (left) and "Reference" (right).
    The same order(s) must be selected (by clicking radio buttons) on both the primary and reference sides of the window.
    The left hand side box headed "Reference flux axis re-scale:" allows rescaling as desired. Then click on "Plot Now" to see the over plotted spectra in the display box.
    Clicking on "Set Plot Style" displays another window "Plot Style Preferences" which allows you to set different line styles for the primary and reference spectra.
    

    Note :

    You cannot return to the IDEA window until you have clicked "Quit".

48. If you leave an application and return to the IDEA window you may be left with an outline box on the IDEA window display box. To remove this click on "Plot".
    Then to re-display the "Applications" window draw another box around the data (using the right mouse key) and "Applications" window will re-appear.
49. If you want to combine the spectra then in the "Applications" window click on the "Arithm2(Ref) button.
50. A new window will appear headed "arithm2(apply the reference to the prime)".
51. Select the "Order" option then under "operation to perform:" select the "mean" radio button.
52. Then click "Apply (Keep)" at the bottom left hand corner of the window.
53. The resultant data displayed in the IDEA Window is the result of co-adding the data from the two different nod positions.
54. At the top left hand corner of the IDEA Window is a small box labeled "Prime". This will now display "arithm2:[filename1]".
55. Click on "Store-Prime" on the top row of buttons on the IDEA Window.
56. Then click "Store". The resultant window will now have a three entry list.
    The last entry will be called "arithm2:[filename1]". This entry is the newly co-added spectrum.
57. Select this entry then click on "Apply function to...". On the sub-menu click on "Change Name".
58. A new window will appear with an entry line "New Title:". Enter a new name, eg: "Extracted_co-added_1". Click "OK".
59. Click "Store". The last entry of the list will be re-named.
60. This co-added spectrum is now held within the IDEA window.
    If you wish to "Quit" the IDEA window at this point proceed to step 69 and save the extracted co-added spectrum to the dataset you are working on.
    You can reload it into the IDEA window at any time as a spectrum by highlighting it on the "Dataset:Datset1" window and clicking "View".
    If you want to send or save the spectrum somewhere else proceed as in steps 61-68 below.
61. On the "Stored Data Sets" window select this re-named entry and click on "Apply function to...".
62. On the sub-menu click "Write to disk in...".
63. Another sub menu will appear with three options :
    
    
    1. FITS format - the standard ".fits" table format.
    2. ASCII format - This leads to 2 further options :
       1. Write the header + All the data
       An ASCII file of data in column format containing all data in the dataset.
       2. Write the header + wave flux stdev and flag data
       An ASCII file of data in column format containing the minimum data needed for re-loading into the IDEA window at a later time.
    3. ISAP-XDR format - The IDL "XDR" format. Only use this to save data you intend to further process ONLY with other IDL programs.
64. Select one of the save options. The choice of format will be determined by the input requirements of any piece of software you intend to use next.
    A small pop-up window will appear. At "Name for [type] file" the default name displayed is the current name for the dataset within the IDEA window.
    The default directory is the one from which SMART was invoked. You can change either of these by typing in "/directory/file".
    (Please use absolute path names for directories.) Then click "OK".
    The entry window and the "Stored Data Sets" window will then disappear.
65. At this point you have extracted and stored the first spectra.
66. You can make a postscript file of the data displayed in the IDEA window display box.
    Click the "Hard Copy" button at the bottom right of the IDEA window. This will display a small pop-up window allowing you to set various print options.
    Set the options : "Color or B/W", the output filename, (including directory, please use absolute path names), title, X and Y labels.
    (The default output filename is "hardcopy.ps" which is seldom meaningful.)
    Then click "OK". A postscript file will be generated and written to the directory you specified in the "Filename" option.
67. You can make postscript output of any other dataset by displaying that in the IDEA window and clicking on "Hard Copy".
    To display any dataset first click on "Store" then select the dataset you want, then make it the prime dataset. (See step 36.)
68. If you have further data in the form of pairs of images (source and nod) you can extract and store them in exactly the same way as the first pair described above.
69. Click the "QUIT" button at the bottom left of the IDEA Window.
70. A small window will appear headed "ISAP Exit", displaying the text "Do you really want to exit?".
Click "Yes".
71. A small window will appear headed "Update Project", displaying the text "Do you want to save your spectral data to the SMART Project Manager?". Click "Yes".
72. In the "Dataset:Dataset1" window a new entries will appear in the file list, with the names they had while in the IDEA window.
73. On the "Dataset:Dataset1" window click "Close".
74. On the Project Manager" window click "Exit".
75. A small pop-up window headed "Question" will appear asking "Save Project ? [project name].
    Click "Yes". If you have loaded an existing project this will re-save it with the additional extracted and co-added data.
76. You have now left SMART.
77. IDL is still invoked at the prompt. Type "exit" at the prompt to leave IDL.