Skip to content

Simulation Ticket Review Procedure

After a package is processed by following the operational procedure, the user assigns the ticket to a member of the CDDS team to review the processing and then perform the final step to submit the data for publication. This document describes the steps that need to be performed in the review. The check before publishing steps should also be done by the user who processed the data to check that the processing has been successful. The final publication steps can only be performed by a member of the CDDS teams with the required permission.

Checklist before publication

Info

All the following checks can be combined in a single command by running: 

. ~cdds/bin/setup_env_for_cdds <version>
cdds_sim_review <location of request.cfg>

  • Check the critical issues log in the critical_issues.log for each component. See the operational procedure for more details.

Info

You can use following command to check for critical logs: 

grep -irn "critical"  $CDDS_PROC_DIR --exclude "*.py" --exclude "*.svn*"

  • Check the QC report in $CDDS_PROC_DIR/qualitycheck/report_<date-time>.json. The aggregated_summary and details fields should be empty.

  • Check approved variables list that expected variables have been added.

  • Check that variables with recently reported issues are not in the approved variables list.

  • Check permissions for proc/archive/log. It should have write permissions for everyone, so move_in_mass can work.

Info

You can check the premissions for proc/archive/log by running following command: 

ls -l $CDDS_PROC_DIR/transfer/ | grep log
The permission should be drwxrwxrwx for the log directory.

  • Look for any partial files left over from the concatenation process using the find command: 
    find $CDDS_DATA_DIR/output/*_* -type f

Submission for publication

In order to complete the submission, you will need access to the els055 server. Only members of the CDDS team are expected to do this part.

  • After completing the pre-publication checks, add a message to the ticket to confirm that the processing was successful.
  • Log on to els055.

Info

If you have not logged on before, please see the section below on setting up els055.

  • Run the move_in_mass command that was generated when running cdds_sim_review.

Example

move_in_mass request.json -p --original_state=embargoed --new_state=available --mass_location=production --variables_list_file=approved_variables_YYYY-MM-DDThhmmss.txt
  • When move_in_mass has completed, check for errors in the log file: $CDDS_PROC_DIR/archive/log/move_in_mass_<date-time>.log

Info

Use grep to search for any CRITICAL messages in the log file: 

grep "CRITICAL" $CDDS_PROC_DIR/archive/log/move_in_mass_<date-time>.log
The <date-time> must be replaced to the time stamp of the log file.

  • Confirm that the last line includes the phrase Moving complete.

  • Check the message queue to confirm that messages are waiting for processing.

Info

  • For CMIP6, you can use following command to confirm that shows the messages are waiting for processing: 
    list_queue CMIP6_available
  • For CMIP6Plus, you can use following command to confirm that shows the messages are waiting for processing: 
    list_queue CMIP6Plus_available
  • Once you are happy that the move_in_mass command has executed successfully, add a message stating this to the ticket, change the status of the ticket to approved and reassign to the original user for teardown and closing.

Setting up els055

  • Create a new file ~/.cdds_credentials.

  • Contact Matt Mizielinski for the required settings.