;+ ; ; AUTHOR: ; Pamme Crandall ; Bill Barrett significant modifications since 2014 ; ; FILENAME: ; get_cips_sci_tlm ; ; DATE: ; 10/31/2005 ; ; PURPOSE: ; retrieve science telemetry for the AIM mission CIPS instrument ; ; INPUT PARAMETERS: ; ; startTime - ; The lower time range for which data will be returned. ; May be specified as julian days, gps microseconds, or ; SORCE mission day numbers. ; ; stopTime - ; The upper time range for which data will be returned ; May be specified as julian days, gps microseconds, or ; SORCE mission day numbers. ; ; cam - ; The camera used to take the image. Possible values: ; 'px', 'mx', 'py', 'my'. ; ; OUTPUT PARAMETERS: ; NONE ;get_cips_sci_tlm ; RETURN VALUE: ; cips_data ; array of cips science data structures ; ; KEYWORD ARGUMENTS: ; sid - ; The system ID for the image. Possible values: ; 1 - SC_FLIGHT ; 2 - SC_TEST ; 3 - FLATSAT ; 4 - ISIM ; ; cycle - ; The cycle for the image; used to differentiate between ; data sets that have the same timestamp; this occurs ; from time to time when the clock is rolled back during testing ; ; rowlimit ; The maximum number of rows to retrieve. If not set or ; less than/equal to zero, do not set limit_rows to rowlimit ; in the query_database call. ; gps ; If set, indicates that input times are to be interpreted as ; microseconds since Jan 6, 1980 midnight UT. (default) ; ; mission_days ; If set, indicates that input times are to be interpreted as ; days elapsed since launch, i.e. mission days. If only the startTime ; argument contains a non-zero value (stopTime is not supplied), then ; all data for the specified mission day number will be retrieved. ; ; julian_days ; If set, indicates that the input times are to be interpreted as ; julian day numbers. Julian dates (abbreviated JD) are a continuous ; count of days and fractions since noon Universal Time on ; January 1, 4713 BCE (on the Julian calendar). Almost 2.5 million ; days have transpired since this date. ; ; lasp_days ; If set, indicates that the input times are to be interpreted as ; lasp day numbers or "LA time," YYYY/DDD-HH:MM:SS ; ; ; obs_id - CIPS observation ID for this science image. ; obs_id Observation Types ; 0x0000 - primary science -- non continuous imaging ; 0xC100 - primary science -- continuing imaging ; 0xD000 - dark -- non continuous imaging ; 0xC1DD - dark -- continuing imaging ; 0x1000 - first light (PX only) ; 0x2000 - last light (not in MX or MY)​ ; 0xF000 - flat field ; 0x5555 - stellar ; 0xDDDD - dummy images (PY only) ; 0x1200 - sub solar ; 0x1212 - nadir pointing subsolars, first taken Sept 27, 07 ; 0x00FC - fast cadence (not in MX or MY) ; 0x12FC - noon fast cadence ; 0x0070 - Science Image Flying forward with TDI ; 0x5070 - Science Image in Snapshot mode ; 0x1070 - Science Image Flying backwards with TDI ; 0x5560 - Science Image in 60 minute snapshot only sequence ; 0x5570 - Science Image in 70 minute snapshot only sequence ; 0x0060 - Science Image Flying forward with TDI ; 0x5060 - Science Image Flying backwards with TDI ; 0xE5EF - pre-flight test (only from 10-May-2006 15:39:30.36 to 10-May-2006 20:16:53.36) ; 0xE4EF - initial test images (only from 15-May-2007 13:37:13.95 to 15-May-2007 13:40:35.28) ; ; img_columns_min - minimum number of columns in the image. ; ; img_columns_max - maximum number of columns in the image. ; ; img_rows_min - minumum number of rows in the image. ; ; img_rows_max - maximum number of rows in the image. ; ; img_row_offset_min - minimum CIPS camera image row start offset from CCD origin. ; ; img_row_offset_max - maximum CIPS camera image row start offset from CCD origin. ; ; img_col_offset_min - minimum CIPS camera image column start offset from CCD origin. ; ; img_col_offset_max - maximum CIPS camera image column start offset from CCD origin. ; ; fast_clear_count_min - minimum CIPS camera fast clear count. ; ; fast_clear_count_max - maximum CIPS camera fast clear count. ; ; tdi_dwell_time_min - minimum CIPS camera TDI dwell time. ; ; tdi_dwell_time_max - maximum CIPS camera TDI dwell time. ; ; tdi_row_count_min - minimum TDI row count setting for this image. ; ; tdi_row_count_max - maximum TDI row count setting for this image. ; ; shutter_close_time_min - minimum CIPS camera shutter close time. ; ; shutter_close_time_max - maximum CIPS camera shutter close time. ; ; frame_sync_time_min - minimum CIPS camera frame sync (delay) time. ; ; frame_sync_time_max - maximum CIPS camera frame sync (delay) time. ; ; image_mode - Camera FPGA imaging mode for data in this science packet. ; ; Compression - Type of compression performed on image data within this packet. ; ; software_binning - Describes if software binning was enabled for this image. ; ; camera_binning - Describes if camera FPGA binning was enabled for this image. ; ; shutter_inhibit - Camera shutter open disable flag. ; ; adc_latchup - CIPS camera ADC latchup status. ; ; pixel_count_error - Error flag set when number of pixels sent by camera FPGA is not correct. ; ; opcode_error - CIPS CAMI FPGA opcode error count. ; ; parity_error - CIPS CAMI FPGA parity error count. ; ; row_sequence_error - CIPS CAMI FPGA row sequence error count. ; ; gps ; If set, indicates that input times are to be interpreted as ; microseconds since Jan 6, 1980 midnight UT. (default) ; ; mission_days ; If set, indicates that input times are to be interpreted as ; days elapsed since launch, i.e. mission days. If only the startTime ; argument contains a non-zero value (stopTime is not supplied), then ; all data for the specified mission day number will be retrieved. ; ; julian_days ; If set, indicates that the input times are to be interpreted as ; julian day numbers. Julian dates (abbreviated JD) are a continuous ; count of days and fractions since noon Universal Time on ; January 1, 4713 BCE (on the Julian calendar). Almost 2.5 million ; days have transpired since this date. ; ; lasp_days ; If set, indicates that the input times are to be interpreted as ; lasp day numbers or "LA time," YYYY/DDD-HH:MM:SS ; ; SUPPORTED DATABASE TABLES OR FILES: ; AIM_CIPS_L1S ; ; EXAMPLE USAGE: ; cips_data = get_cips_sci_tlm(8.2008166e+14,8.2872192e+14, 'mx', rowlimit=200, /gps) ; cips_data = get_cips_sci_tlm('2006/000-16:27:31', '2006/100-16:31:47', 'mx', /lasp_days) ; ; ; VERSION: ; version 1.4 7/17/06 ;- ; MODIFICATION HISTORY: ; 11/21/07 in aim_uncompress_img changed the cast of compressed_data to ; long64 from long because fill values were being truncated. ; ; 10/01/07 modified byteorder call to account for bytes_per_num ; ; 10/01/07 pc Changed the default value for sid to 1 for flight. ; ; 5/15/07 replaced min_obs_id and max_obs_id with obs_id ; ; 3/15/07 added a descending_order keyword to reverse sort images over a ; provided time range ; ; 8/30/06 changed method for determining bytes_per_num to image_length / ; n_rows / n_cols so that if the method for determining data type ; changes in the ingest, it won't affect the operation of this query ; ; 7/17/06 added aim_uncompress_img function to this file PLC ; ; 6/26/06 changed input parameters "sid" and "cycle" to keywords "sid" and "cycle" ; and made the default values 2 and 1, respectively PLC ; ; 3/9/06 changed the method used to determine the image data type PLC ; ; 5/6/06 Karie Shipley: added sid & cycle as inputs, and rowlimit as optional. ; Also queries for the number of rows and maximum image size that would ; be returned from the full query, and uses it for max_image_length. ; ;------------------------------------------------------------------------------- ; PURPOSE: ; to return an array of uncompressed data ; ; INPUT VARIABLE ; compressed_data - array of data compressed using root mean squared ; compression algorithm ;------------------------------------------------------------------------------- function aim_uncompress_img, compressed_data ;uncompress all data uncompressed_data = ((((long64(compressed_data)) - 511)^2)/2) ;for data elements that were originally less than ;or equal to 543, return them to their initial value no_change_index = where(compressed_data le 543) if n_elements(no_change_index) gt 0 then begin if no_change_index[0] ne -1 then begin no_change_data = compressed_data[no_change_index] uncompressed_data[no_change_index] = no_change_data endif endif return, uncompressed_data end ;------------------------------------------------------------------------------- ;------------------------------------------------------------------------------- function get_cips_sci_tlm, $ start_time, $ stop_time, $ cam, $ sid=sid, $ cycle=cycle, $ rowlimit=rowlimit, $ gps=gps, $ mission_days = mission_days, $ julian_days = julian_days, $ lasp_days = lasp_days, $ shutter_open_timestamp_min = shutter_open_timestamp_min, $ shutter_open_timestamp_max = shutter_open_timestamp_max, $ shutter_close_timestamp_min = shutter_close_timestamp_min, $ shutter_close_timestamp_max = shutter_close_timestamp_max, $ integration_time_min = integration_time_min, $ integration_time_max = integration_time_max, $ mid_image_timestamp_min = mid_image_timestamp_min, $ mid_image_timestamp_max = mid_image_timestamp_max, $ obs_id=obs_id, $ img_columns_min=img_columns_min, $ img_columns_max=img_columns_max, $ img_rows_min=img_rows_min, $ img_rows_max=img_rows_max, $ img_row_offset_min=img_row_offset_min, $ img_row_offset_max=img_row_offset_max, $ img_col_offset_min=img_col_offset_min, $ img_col_offset_max=img_col_offset_max, $ fast_clear_count_min=fast_clear_count_min, $ fast_clear_count_max=fast_clear_count_max, $ tdi_dwell_time_min=tdi_dwell_time_min, $ tdi_dwell_time_max=tdi_dwell_time_max, $ tdi_row_count_min=tdi_row_count_min, $ tdi_row_count_max=tdi_row_count_max, $ shutter_close_time_min=shutter_close_time_min, $ shutter_close_time_max=shutter_close_time_max, $ frame_sync_time_min=frame_sync_time_min, $ frame_sync_time_max=frame_sync_time_max, $ image_mode=image_mode, $ compression=compression, $ software_binning=software_binning, $ camera_binning=camera_binning, $ shutter_inhibit=shutter_inhibit, $ adc_latchup=adc_latchup, $ pixel_count_error=pixel_count_error, $ opcode_error=opcode_error, $ parity_error=parity_error, $ row_sequence_error=row_sequence_error, $ descending_order=descending_order, $ debug=debug ; ;------------------------------------------------------------------------------ if n_elements(start_time) le 0 or n_elements(stop_time) le 0 then begin doc_library, 'get_cips_sci_tlm' return, ' ' endif if n_elements(sid) eq 0 then sid = 1 if n_elements(cycle) eq 0 then cycle = 1 ; determine the table to query table = 'AIM_CIPS_L1S.CipsScienceTlm' + strupcase(cam) ; determine what type of time format is specified if (keyword_set(julian_days)) then begin ; user specified julian days. gps1 = double(jd2gps(start_time)) gps2 = double(jd2gps(stop_time)) endif else if (keyword_set(mission_days)) then begin gps1 = double(ad2usec(start_time)) gps2 = double(ad2usec(stop_time)) endif else if (keyword_set(gps)) then begin ; user specified gp microseconds. gps1 = double(start_time) gps2 = double(stop_time) endif else if (keyword_set(lasp_days)) then begin ; user specified lasp days gps1 = double(la2gps(string(start_time))) * 1.d6 gps2 = double(la2gps(string(stop_time))) * 1.d6 endif else begin ;default print, get_routine_name() + ' ' + string(start_time) gps1 = double(start_time) gps2 = double(stop_time) endelse ;convert time to a string vtcw_start = string(gps1, FORMAT='(f19.2)') vtcw_stop = string(gps2, FORMAT='(f19.2)') ;build query where_clause = ' where image_tlm_timestamp >= ' + vtcw_start + $ ' and image_tlm_timestamp <= ' + vtcw_stop + $ ' and SID = ' + strtrim(sid,2) + $ ' and SCT_Cycle = ' + strtrim(cycle,2) if n_elements(shutter_open_timestamp_min) gt 0 then $ where_clause = where_clause + ' and shutter_open_timestamp >= ' + strtrim(shutter_open_timestamp_min,2) if n_elements(shutter_open_timestamp_max) gt 0 then $ where_clause = where_clause + ' and shutter_open_timestamp <= ' + strtrim(shutter_open_timestamp_max,2) if n_elements(shutter_close_timestamp_min) gt 0 then $ where_clause = where_clause + ' and shutter_close_timestamp >= ' + strtrim(shutter_close_timestamp_min,2) if n_elements(shutter_close_timestamp_max) gt 0 then $ where_clause = where_clause + ' and shutter_close_timestamp <= ' + strtrim(shutter_close_timestamp_max,2) if n_elements(integration_time_min) gt 0 then $ where_clause = where_clause + ' and integration_time >= ' + strtrim(integration_time_min,2) if n_elements(integration_time_max) gt 0 then $ where_clause = where_clause + ' and integration_time <= ' + strtrim(integration_time_max,2) if n_elements(mid_image_timestamp_min) gt 0 then $ where_clause = where_clause + ' and mid_image_timestamp >= ' + strtrim(mid_image_timestamp_min,2) if n_elements(mid_image_timestamp_max) gt 0 then $ where_clause = where_clause + ' and mid_image_timestamp <= ' + strtrim(mid_image_timestamp_max,2) if n_elements(img_columns_min) gt 0 then $ where_clause = where_clause + ' and img_cols >= ' + strtrim(img_columns_min,2) if n_elements(img_columns_max) gt 0 then $ where_clause = where_clause + ' and img_cols <= ' + strtrim(img_columns_max,2) if n_elements(img_rows_min) gt 0 then $ where_clause = where_clause + ' and img_rows >= ' + strtrim(img_rows_min,2) if n_elements(img_rows_max) gt 0 then $ where_clause = where_clause + ' and img_rows <= ' + strtrim(img_rows_max,2) if n_elements(img_row_offset_min) gt 0 then $ where_clause = where_clause + ' and img_offset_row >= ' + strtrim(img_row_offset_min,2) if n_elements(img_row_offset_max) gt 0 then $ where_clause = where_clause + ' and img_offset_row <= ' + strtrim(img_row_offset_max,2) if n_elements(img_col_offset_min) gt 0 then $ where_clause = where_clause + ' and img_offset_col >= ' + strtrim(img_col_offset_min,2) if n_elements(img_col_offset_max) gt 0 then $ where_clause = where_clause + ' and img_offset_col <= ' + strtrim(img_col_offset_max,2) if n_elements(fast_clear_count_min) gt 0 then $ where_clause = where_clause + ' and fast_clears >= ' + strtrim(fast_clear_count_min,2) if n_elements(fast_clear_count_max) gt 0 then $ where_clause = where_clause + ' and fast_clears <= ' + strtrim(fast_clear_count_max,2) if n_elements(tdi_dwell_time_min) gt 0 then $ where_clause = where_clause + ' and tdi_dwell >= ' + strtrim(tdi_dwell_time_min,2) if n_elements(tdi_dwell_time_max) gt 0 then $ where_clause = where_clause + ' and tdi_dwell <= ' + strtrim(tdi_dwell_time_max,2) if n_elements(tdi_row_count_min) gt 0 then $ where_clause = where_clause + ' and tdi_rows >= ' + strtrim(tdi_row_count_min,2) if n_elements(tdi_row_count_max) gt 0 then $ where_clause = where_clause + ' and tdi_rows <= ' + strtrim(tdi_row_count_max,2) if n_elements(shutter_close_time_min) gt 0 then $ where_clause = where_clause + ' and shutter_cl_time >= ' + strtrim(shutter_close_time_min,2) if n_elements(shutter_close_time_max) gt 0 then $ where_clause = where_clause + ' and shutter_cl_time <= ' + strtrim(shutter_close_time_max,2) if n_elements(frame_sync_time_min) gt 0 then $ where_clause = where_clause + ' and frame_sync_time >= ' + strtrim(frame_sync_time_min,2) if n_elements(frame_sync_time_max) gt 0 then $ where_clause = where_clause + ' and frame_sync_time <= ' + strtrim(frame_sync_time_max,2) if n_elements(image_mode) gt 0 then $ where_clause = where_clause + ' and img_mode = ' + strtrim(image_mode,2) if n_elements(compression) gt 0 then $ where_clause = where_clause + ' and comp_scheme = ' + strtrim(compression,2) if n_elements(software_binning) gt 0 then $ where_clause = where_clause + ' and sw_bin_ena = ' + strtrim(software_binning,2) if n_elements(camera_binning) gt 0 then $ where_clause = where_clause + ' and cam_bin_ena = ' + strtrim(camera_binning,2) if n_elements(shutter_inhibit) gt 0 then $ where_clause = where_clause + ' and shutter_disable = ' + strtrim(shutter_inhibit,2) if n_elements(adc_latchup) gt 0 then $ where_clause = where_clause + ' and adc_lup = ' + strtrim(adc_latchup,2) if n_elements(pixel_count_error) gt 0 then $ where_clause = where_clause + ' and pixel_count_err = ' + strtrim(pixel_count_error,2) if n_elements(opcode_error) gt 0 then $ where_clause = where_clause + ' and opcode_err_ct = ' + strtrim(opcode_error,2) if n_elements(parity_error) gt 0 then $ where_clause = where_clause + ' and parity_err_ct = ' + strtrim(parity_error,2) if n_elements(row_sequence_error) gt 0 then $ where_clause = where_clause + ' and row_seq_err_ct = ' + strtrim(row_sequence_error,2) ; Convert an observation id in the form '0x...' into something easier for Oracle to work with if n_elements(obs_id) gt 0 then begin split_obs_id = (strtrim(string(obs_id), 2)).split('x') obs_id_fixed = 0 reads, split_obs_id[1], obs_id_fixed, format='(z)' where_clause = where_clause + ' and obs_id = ' + strtrim(string(obs_id_fixed), 2) endif get_database_login, 'AIM_CIPS_L1S', user=user, password=password, url=url query_database, user=user, password=password, dbUrl=url, /dbConnect select = 'select ' count_query = select + ' COUNT(*), MAX(LENGTH(image)) from ' + table + where_clause query_database, count_query, data, nrows, user=user, password=password, dbUrl=url if nrows le 0 || data.column_0 le 0 then begin if (obs_id ne '0x1200' && obs_id ne '0x1212' && obs_id ne '0x00FC' && obs_id ne '0x12FC') || $ get_debug_level() gt 5 then print, get_routine_name() + ' no data found from: ' + table + where_clause return, 0 endif count = data.column_0 max_image_length = data.column_1 ; Try to get the data from the database select_stmt = select + ' ' + table + '.*, LENGTH(image) as imagelength from ' + table query = select_stmt + where_clause + ' ORDER BY image_tlm_timestamp' if keyword_set(descending_order) then query=query + ' DESC' ; If a row limit is specified it must be gt 0 ; A negative row limit is a flag to query database to return all available rows rowlimit = (keyword_set(rowlimit) && rowlimit gt 0) ? rowlimit : -1 query_database, query, data, nrows, user=user, password=password, dbUrl=url, limit_rows=rowlimit ; If this is not a request for a calibration record ('0x1200', etc.) if (obs_id ne '0x1200' && obs_id ne '0x1212' && obs_id ne '0x00FC' && obs_id ne '0x12FC') || $ get_debug_level() gt 5 then print, get_routine_name() + ' query: ' + table + where_clause if get_debug_level() gt 5 then begin ; Print the results of the query, and return if no rows were returned if nrows le 0 || size(data, /type) ne 8 then begin print, get_routine_name() + ' no data found for tlm (obs) id = ' + obs_id return, 0 endif else print, get_routine_name() + ' tlm (obs) id = ' + obs_id + ' ' + strtrim(string(nrows),2) + $ ', row' + (nrows gt 1 ? 's' : '') + ', max image length = ' + strtrim(string(max_image_length),2) endif ;copy data struct to cips_data struct, which includes all the fields but ;the image, imagelength and image_length_ cips_data_struct = get_cips_data_struct() cips_data = replicate(cips_data_struct, n_elements(data)) struct_assign, data, cips_data ;reduce image to its real size, unzip it, uncompress it, ;make it 2-dimensional and create a pointer to it from the image ;field of the cips_data structure for i=0L, n_elements(data)-1 do begin if data[i].imagelength le 0 then continue resized_image = (data[i].image)[0:data[i].imagelength - 1] ;change from byte array to either integer or long array ;based on bits_per_pixel ;if data[i].bits_per_pixel ge 16 then bytes_per_num = 4 else bytes_per_num = 2 ;if long(data[i].img_cols) * long(data[i].img_rows) * bytes_per_num $ ; ne data[i].imagelength then continue bytes_per_num = data[i].imagelength / long(data[i].img_cols) / long(data[i].img_rows) num_img_elements = long(data[i].img_cols) * long(data[i].img_rows) case bytes_per_num of 4 : begin byteorder, resized_image, /SWAP_IF_LITTLE_ENDIAN , /lswap corrected_data_type_image = long(resized_image, 0, num_img_elements) end 2 : begin byteorder, resized_image, /SWAP_IF_LITTLE_ENDIAN corrected_data_type_image = uint(resized_image, 0, num_img_elements) end endcase if cips_data[i].comp_scheme eq 1 then uncomp_image = aim_uncompress_img(corrected_data_type_image) $ else uncomp_image = corrected_data_type_image two_dim_image = reform(uncomp_image, data[i].img_cols, data[i].img_rows) cips_data[i].image = ptr_new(float(two_dim_image)) ;Take care of timestamps its=cips_data[i].image_tlm_timestamp it=cips_data[i].integration_time s_cl_ts=cips_data[i].shutter_close_timestamp s_op_ts=cips_data[i].shutter_open_timestamp m_ts=cips_data[i].mid_image_timestamp int_in_usec=cips_data[i].integration_time gt 2000.0d if ~int_in_usec then begin it*=1000.0d cips_data[i].integration_time=it endif consistent=abs((s_cl_ts-s_op_ts)-it) lt 10.0d if ~consistent then begin cips_data[i].shutter_close_timestamp=its+1000.0d*(s_cl_ts-its) cips_data[i].shutter_open_timestamp=its+1000.0d*(s_op_ts-its) endif s_cl_ts=cips_data[i].shutter_close_timestamp s_op_ts=cips_data[i].shutter_open_timestamp cips_data[i].mid_image_timestamp=(s_cl_ts+s_op_ts)/2 endfor return, cips_data end