25.1.1

GOATS 25.1.1 (2025-01-30)

New Features

• Add delete run functionality: Enabled a delete button for DRAGONS runs, allowing users to reclaim disk space. Extended the API to support run deletions. [#GOATS-428]
• Added TNS query support: Developed class to query TNS objects and return payload. [#GOATS-574]
• Updated TNS harvester: Modified harvester to use the TNSClient for object querying. [#GOATS-575]
• Added LICENSE to repository. [#GOATS-151]
• Add default recipe card with instructions: Introduced a default card that guides users to select a recipe. Provides tips on starting and stopping DRAGONS reduction, modifying recipes, and viewing logs. [#GOATS-370]
• Show processed files in run directory: Renamed "Output Files" to "Processed Files" across classes and objects. Added button to view files in JS9 and display headers in a modal. Introduced DataProductMetadata model to minimize astrodata reads. [#GOATS-429]
• Added user docs button: Added a button to the navbar that opens the user documentation in a new tab. [#GOATS-495]
• Improved facility status page: Fetches and displays Gemini North and South status and updated weather URLs. [#GOATS-497]
• Add filesearch textbox in Manage Data: Enhanced file management with a search box to filter files by filename and path. [#GOATS-515]
• Improve cancel functionality: Enabled multiple attempts to stop background tasks during DRAGONS reduction if the initial cancellation fails. [#GOATS-524]
• Fetch initial running reductions: Added functionality to retrieve and display initial running reductions on the DRAGONS page. Users can now see the current status of reductions immediately upon page load. [#GOATS-525]
• Added responsive table format for long Target values in view. [#GOATS-530]
• Added calibration file viewing and header display: Implemented support for viewing calibration files through the DRAGONS interface with JS9 and displaying FITS header information. [#GOATS-555]

Changes

• Used local fontawesomefree: Incorporated FontAwesome into GOATS static assets and removed external Python dependency. [#GOATS-535]
• Removed Update Broker Data button: Removed the "Update Broker Data" button from the target list view. [#GOATS-160]
• Refactored product IDs: Changed how products are stored by using file paths to handle files in different directories with the same product IDs. [#GOATS-473]
• Updated environment.yaml for latest DRAGONS: Updated the environment file to include the latest DRAGONS release with patches specific to GOATS. [#GOATS-547]
• Remove tom-antares dependency: Ingested its functionality directly into GOATS due to extensive customizations and installation complexities. [#GOATS-554]

Bug Fixes

• Fixed file deletion bug: Correctly built full path for processed files to delete. [#GOATS-559]
• Handle duplicate file entries in checksum files: Fixed an issue where duplicate file entries in GOA checksum files caused errors during downloading and decompression. The process now skips duplicates and continues without interruption. [#GOATS-577]
• Set astroquery version: Fixed SIMBAD query compatibility by pinning astroquery to a working version. [#GOATS-579]
• Fixed calibration path handling: Resolved issue with spaces in calibration database paths causing errors during DRAGONS reduction. [#GOATS-317]
• Fixed ANTARES queries: Ensured user queries can be renamed properly and querying with elastic search works. [#GOATS-498]
• Fix issue with conda environment with GitHub Actions. [#GOATS-504]
• Added functionality to handle decompression of bz2 FITS files uploaded into the calibration database. Previously, silent errors occurred due to improper handling of decompression and file placement. [#GOATS-556]
• Workaround for DRAGONS version mismatch: Addressed an issue where the DRAGONS version reported by pip differed from the conda-installed version by implementing logic to pull the version directly from conda. [#GOATS-557]

What's Changed

• GOATS-151: Add license to repository. by @davner in #228
• GOATS-495: Add button to navbar for user documentation. by @davner in #229
• GOATS-160: Remove "Update Broker Data" button from target list view. by @davner in #230
• GOATS-530: Make Target view table responsive with scrollbar for long values. by @davner in #231
• GOATS-473: Store products with paths to avoid clashes in product IDs. by @davner in #232
• GOATS-498: Fix querying and renaming with ANTARES plugin. by @davner in #233
• GOATS-497: Add ability to fetch facility status of Gemini North and South and update weather URLs. by @davner in #234
• GOATS-524: Improve cancel functionality for DRAGONS reduction. by @davner in #235
• GOATS-525: Fetch initial running reductions on DRAGONS page. by @davner in #236
• GOATS-544: Remove unnecessary bpm file. by @davner in #237
• GOATS-370: Add default recipe card with instructions and tips for DRAGONS reduction. by @davner in #238
• GOATS-515: Add filesearch textbox in Manage Data. by @davner in #239
• GOATS-371: Fix bug with spaces in calibration database paths. by @davner in #240
• GOATS-554: Remove tom-antares dependency. by @davner in #241
• GOATS-429: Show processed files in run directory. by @davner in #242
• GOATS-504: Bugfix for conda environment for GitHub Actions. by @davner in #243
• GOATS-504: Direct to miniforge3 variant. by @davner in #244
• GOATS-555 GOATS-556: JS9/Header viewer for caldb and bugfix with bz2decompression and uploaded caldb files. by @davner in #245
• GOATS-547 GOATS-557: Update environment.yml file for alpha and workaround for pip vs conda DRAGONS version. by @davner in #246
• GOATS-559: Fix file deletion bug for processed files. by @davner in #247
• GOATS-535: Use local fontawesome assets. by @davner in #248
• GOATS-574: Add TNS query class for object information. by @davner in #249
• GOATS-575: Modify TNS harvester to use TNSClient. by @davner in #250
• GOATS-579: Set astroquery version to ensure SIMBAD query compatibility. by @davner in #251
• GOATS-577: Handle duplicate file entries in checksum files. by @davner in #252
• GOATS-428: Add delete run functionality to DRAGONS. by @davner in #253

Full Changelog: 24.12.0...25.1.1

25.1.0-alpha.1

GOATS 25.1.0-alpha.1 (2025-01-25)

New Features

• Added TNS query support: Developed class to query TNS objects and return payload. [#GOATS-574]
• Updated TNS harvester: Modified harvester to use the TNSClient for object querying. [#GOATS-575]

Changes

• Used local fontawesomefree: Incorporated FontAwesome into GOATS static assets and removed external Python dependency. [#GOATS-535]

Bug Fixes

• Fixed file deletion bug: Correctly built full path for processed files to delete. [#GOATS-559]
• Handle duplicate file entries in checksum files: Fixed an issue where duplicate file entries in GOA checksum files caused errors during downloading and decompression. The process now skips duplicates and continues without interruption. [#GOATS-577]
• Set astroquery version: Fixed SIMBAD query compatibility by pinning astroquery to a working version. [#GOATS-579]

What's Changed

• GOATS-559: Fix file deletion bug for processed files. by @davner in #247
• GOATS-535: Use local fontawesome assets. by @davner in #248
• GOATS-574: Add TNS query class for object information. by @davner in #249
• GOATS-575: Modify TNS harvester to use TNSClient. by @davner in #250
• GOATS-579: Set astroquery version to ensure SIMBAD query compatibility. by @davner in #251
• GOATS-577: Handle duplicate file entries in checksum files. by @davner in #252

Full Changelog: 25.1.0-alpha.0...25.1.0-alpha.1

25.1.0-alpha.0

GOATS 25.1.0-alpha.0 (2025-01-14)

New Features

• Added LICENSE to repository. [#GOATS-151]
• Add default recipe card with instructions: Introduced a default card that guides users to select a recipe. Provides tips on starting and stopping DRAGONS reduction, modifying recipes, and viewing logs. [#GOATS-370]
• Show processed files in run directory: Renamed "Output Files" to "Processed Files" across classes and objects. Added button to view files in JS9 and display headers in a modal. Introduced DataProductMetadata model to minimize astrodata reads. [#GOATS-429]
• Added user docs button: Added a button to the navbar that opens the user documentation in a new tab. [#GOATS-495]
• Improved facility status page: Fetches and displays Gemini North and South status and updated weather URLs. [#GOATS-497]
• Add filesearch textbox in Manage Data: Enhanced file management with a search box to filter files by filename and path. [#GOATS-515]
• Improve cancel functionality: Enabled multiple attempts to stop background tasks during DRAGONS reduction if the initial cancellation fails. [#GOATS-524]
• Fetch initial running reductions: Added functionality to retrieve and display initial running reductions on the DRAGONS page. Users can now see the current status of reductions immediately upon page load. [#GOATS-525]
• Added responsive table format for long Target values in view. [#GOATS-530]
• Added calibration file viewing and header display: Implemented support for viewing calibration files through the DRAGONS interface with JS9 and displaying FITS header information. [#GOATS-555]

Changes

• Removed Update Broker Data button: Removed the "Update Broker Data" button from the target list view. [#GOATS-160]
• Refactored product IDs: Changed how products are stored by using file paths to handle files in different directories with the same product IDs. [#GOATS-473]
• Updated environment.yaml for latest DRAGONS: Updated the environment file to include the latest DRAGONS release with patches specific to GOATS. [#GOATS-547]
• Remove tom-antares dependency: Ingested its functionality directly into GOATS due to extensive customizations and installation complexities. [#GOATS-554]

Bug Fixes

• Fixed calibration path handling: Resolved issue with spaces in calibration database paths causing errors during DRAGONS reduction. [#GOATS-317]
• Fixed ANTARES queries: Ensured user queries can be renamed properly and querying with elastic search works. [#GOATS-498]
• Fix issue with conda environment with GitHub Actions. [#GOATS-504]
• Added functionality to handle decompression of bz2 FITS files uploaded into the calibration database. Previously, silent errors occurred due to improper handling of decompression and file placement. [#GOATS-556]
• Workaround for DRAGONS version mismatch: Addressed an issue where the DRAGONS version reported by pip differed from the conda-installed version by implementing logic to pull the version directly from conda. [#GOATS-557]

What's Changed

• GOATS-151: Add license to repository. by @davner in #228
• GOATS-495: Add button to navbar for user documentation. by @davner in #229
• GOATS-160: Remove "Update Broker Data" button from target list view. by @davner in #230
• GOATS-530: Make Target view table responsive with scrollbar for long values. by @davner in #231
• GOATS-473: Store products with paths to avoid clashes in product IDs. by @davner in #232
• GOATS-498: Fix querying and renaming with ANTARES plugin. by @davner in #233
• GOATS-497: Add ability to fetch facility status of Gemini North and South and update weather URLs. by @davner in #234
• GOATS-524: Improve cancel functionality for DRAGONS reduction. by @davner in #235
• GOATS-525: Fetch initial running reductions on DRAGONS page. by @davner in #236
• GOATS-544: Remove unnecessary bpm file. by @davner in #237
• GOATS-370: Add default recipe card with instructions and tips for DRAGONS reduction. by @davner in #238
• GOATS-515: Add filesearch textbox in Manage Data. by @davner in #239
• GOATS-371: Fix bug with spaces in calibration database paths. by @davner in #240
• GOATS-554: Remove tom-antares dependency. by @davner in #241
• GOATS-429: Show processed files in run directory. by @davner in #242
• GOATS-504: Bugfix for conda environment for GitHub Actions. by @davner in #243
• GOATS-504: Direct to miniforge3 variant. by @davner in #244
• GOATS-555 GOATS-556: JS9/Header viewer for caldb and bugfix with bz2decompression and uploaded caldb files. by @davner in #245
• GOATS-547 GOATS-557: Update environment.yml file for alpha and workaround for pip vs conda DRAGONS version. by @davner in #246

Full Changelog: 24.12.0...25.1.0-alpha.0

24.12.0

GOATS 24.12.0 (2024-12-21)

New Features

• Implemented dataproduct visualizer template tag: Designed and implemented a templatetag to fetch and display dataproducts for visualization based on data type. [#GOATS-489]
• Add photometric data plotting: Refactored plotting logic and enhanced interface usability. [#GOATS-490]
• Added tests for API endpoints added for data visualizer. [#GOATS-492]
• Connected backend API with frontend fetching: Implemented async fetching to dynamically retrieve or process dataproducts for plotting. [#GOATS-493]
• Added Plotly.js for dynamic plotting: Integrated Plotly.js for interactive plotting in the dataproduct visualizer and implemented styling to toggle between dark and light themes. [#GOATS-494]
• Added django filter for reduced dataproducts: Allowed querying of reduced data by product ID and data type. [#GOATS-496]
• Added plotting function to update plot with requested spectroscopy data. [#GOATS-499]
• Extended Gemini facility class functionality: Added methods for reading FITS headers and handling Gemini-specific image data. [#GOATS-503]
• Added search field for file names: Implemented client-side filtering for the File Name column on the data visualizer to allow users to quickly find files. [#GOATS-509]
• Update plot with axis unit handling and editable labels: Added support to display correct units for Wavelength and Flux if available in FITS files. Defaulted to "Wavelength" and "Flux" when units are missing. Made axis labels editable for manual input with CSV files for both photometry and spectroscopy. [#GOATS-510]
• Added editable axis ranges: Enabled users to click directly on x and y axis end values to edit their ranges. [#GOATS-511]
• Added user feedback when no files matched filter criteria during file plotting. [#GOATS-512]

Changes

• Update photometry tab message: Revised message to include supported CSV format with a link to Manage Data. [#GOATS-507]
• Update spectroscopy tab message: Revised message to include supported FITS and CSV formats with a link to Manage Data. [#GOATS-508]

Bug Fixes

• Dynamic WebSocket URL generation: Built WebSocket URL from window and request. [#GOATS-281]
• Converted endpoint to API: Browser extension endpoint now functions as a fully integrated API endpoint with proper token authentication to validate posts. [#GOATS-383]
• Fixed issue with Django template and airmass plot. [#GOATS-500]
• Fixed typo with filter backend in the settings template. [#GOATS-501]
• Implemented workaround for CORS-related issue with plotting. [#GOATS-502]
• Fixed issue with url for fetching and plotting data. [#GOATS-505]

What's Changed

• GOATS-383: Convert browser extension endpoint to API. by @davner in #210
• GOATS-496: Add django filter for reduced dataproducts. by @davner in #211
• GOATS-489: Implement dataproduct visualizer template tag. by @davner in #212
• GOATS-494: Add Plotly.js for dynamic plotting in visualizer. by @davner in #213
• GOATS-493: Connect backend API with frontend for dynamic fetching. by @davner in #214
• GOATS-499: Add ability to update plot with requested spectroscopy data. by @davner in #215
• GOATS-490: Add photometric data plotting interface. by @davner in #216
• GOATS-500: Fix issue with django template and airmass plot. by @davner in #217
• GOATS-501: Fix issue with typo in settings template with filter backends. by @davner in #218
• GOATS-492 GOATS-503: Add tests for data visualizer and extend Gemini facility. by @davner in #219
• GOATS-502: Implement workaround for CORS-issue during development. by @davner in #220
• GOATS-505: Change url to relative url when plotting. by @davner in #221
• GOATS-281: Build WebSocket URL dynamically from window and request. by @davner in #222
• GOATS-507 GOATS-508: Update info message for spectroscopy and photometry tab. by @davner in #223
• GOATS-510: Update plot with axis unit handling and editable labels. by @davner in #224
• GOATS-509: Add search field for file names on data visualizer. by @davner in #225
• GOATS-512: Add user feedback for no file matches. by @davner in #226
• GOATS-511: Add ability to edit x and y axis ranges. by @davner in #227

Full Changelog: 24.11.0...24.12.0

24.11.0

GOATS 24.11.0 (2024-11-27)

New Features

• Added navbar to observation page: Implemented a new template tag to include the navigation bar on the observation page for targets. [#GOATS-173]
• Added GHOST in DRAGONS application: Implemented features in DRAGONS application to debundle and reduce GHOST data. Bugfix for file group selection and improved astroquery login verification. [#GOATS-328]
• Enhanced file fetch control: Added a checkbox to the UI that allows users to fetch all files for an observation ID, disabling the default filters of observation class, type, and object name. This change grants users full control over the selection of files for use in DRAGONS recipe reductions. [#GOATS-411]
• Renamed 'uparms' for clarity and added a tooltip to assist users in using it correctly. [#GOATS-444]
• Added API endpoint for DRAGONS reduced images: Implemented a new processor to extract data from DRAGONS reduced images and extended TOMToolkit functions to support new requirements. [#GOATS-484]

Changes

• Refactored codebase for better organization. [#GOATS-329]
• Removed unnecessary data types for data products: Only 'fits_file' is needed for DRAGONS reduction. [#GOATS-445]
• Hide UI elements without run selection: The visibility of the output files and calibration database manager is now controlled by the selection of a run ID. [#GOATS-467]
• Sort files by observation type for DRAGONS compatibility: Ensured the first file in the list matches the recipe's observation type to prevent mismatches with tags and primitives. [#GOATS-479]

Bug Fixes

• Fixed observation record ID handling: Corrected an issue where a hardcoded observation ID from testing persisted into production, ensuring that only runs associated with an actual observation record are displayed. [#GOATS-464]
• Fixed filter expression and ID uniqueness bugs: Resolved an issue where user-provided filter expressions were not correctly used in filtering and grouping available files. Additionally, improved the uniqueness of file checkbox IDs by incorporating more identifying information, addressing an issue uncovered when allowing user access to all files. [#GOATS-465]
• Fixed recipe and primitive extraction for DRAGONS application: Extracted primitives now include all lines, ensuring comments and docstrings are no longer ignored. [#GOATS-470]
• Added safeguard for missing primitive params in showpars: Ensured DRAGONS/GOATS showpars handles cases where parameters for specific primitives are absent. [#GOATS-471]
• Fixed query order operations: Corrected handling of logical operations in expressions. Implemented using the ast module to parse expressions more reliably. Updated logical operators to be case-sensitive as required by ast. Removed "not" but added "!=" as a valid operation. Updated UI help documentation to reflect these changes. [#GOATS-474]
• Bugfix for numerical astrodata descriptors: Allowed numerical values for astrodata_descriptors like 'object'. Users now need to enclose strings in quotes for correct parsing, while numerical values should be entered without quotes. Added a default return to ensure consistent API responses when no files are found during grouping. [#GOATS-475]

What's Changed

• GOATS-411: Add checkbox for unrestricted file fetching. by @davner in #195
• GOATS-465: Fix filter expression usage and improve checkbox ID uniqueness. by @davner in #196
• GOATS-464: Remove hardcoded observation ID. by @davner in #197
• GOATS-467: Hide UI sections without selected run. by @davner in #198
• GOATS-444: Rename 'uparms' for clarity. by @davner in #199
• GOATS-329: Refactor codebase and clean up directories. by @davner in #200
• GOATS-328: Added GHOST data processing. by @davner in #201
• GOATS-470: Fix extraction to include comments and docstrings. by @davner in #202
• GOATS-471: Add safeguard for missing primitive params for help docs. by @davner in #203
• GOATS-475: Ensure correct parsing for numerical descriptors. by @davner in #204
• GOATS-474: Fixed query order operations. by @davner in #205
• GOATS-479: Sort files by observation type to match DRAGONS expectations. by @davner in #206
• GOATS-484: Add API endpoint for DRAGONS reduced images. by @davner in #207
• GOATS-173: Add navbar to observation page. by @davner in #208
• GOATS-445: Remove unnecessary data types. by @davner in #209

Full Changelog: 24.10.0...24.11.0

24.10.0

GOATS 24.10.0 (2024-10-29)

New Features

• Added API backend for output file listing: Implemented functionality to list output files and their last modified timestamps from a DRAGONSRun. [#GOATS-426]
• Linked API with UI for output directory display: Integrated the API and UI to enhance visibility of the output file directory. Added user feedback mechanisms for updates and refresh actions. [#GOATS-430]
• Added API file management for DRAGONS runs: Extended the system to allow adding files from the output directory of a DRAGONS run to the saved dataproducts. Users can now also remove these files; doing so deletes both the dataproduct entry and the file itself. [#GOATS-431]
• Linked backend and frontend for DRAGONS output file operations: The integration now allows adding output files to data products and removing them directly through the frontend interface. [#GOATS-433]
• Designed uparms UI for DRAGONS recipe modification: Implemented a user interface to edit 'uparms' for recipes, requiring 'edit' mode activation similar to existing recipe and primitive modifications. [#GOATS-434]
• Extended DRAGONS recipe "uparms" handling in API: Updated the backend to support modifications to "uparms" for DRAGONS recipe reductions. The update includes parsing "uparms" from string format into Python objects, enabling dynamic parameter adjustments. [#GOATS-435]
• Connected frontend to backend for using uparms in DRAGONS reduction. [#GOATS-436]
• Refactored DRAGONS logger: Improved efficiency and cleaned up code. [#GOATS-437]
• Refactored progress bar for recipes: Improved maintainability and readability of the code handling the recipe progress bar. [#GOATS-438]
• Fix versioning issues: Resolved bugs in tomtoolkit, GOA, and astroquery. Fixed tomtoolkit version to prevent future compatibility issues. [#GOATS-439]

Changes

• Major refactor of DRAGONS app: Accommodated changes to recipe and file nesting. [#GOATS-412]
• Refactor run panel UI: Improved loading animation and user feedback during actions. [#GOATS-441]
• Refactored files table: Improved display of groups and file toggling for runs. [#GOATS-442]
• Moved API to singleton design: Simplified DRAGONS API by converting it to a singleton pattern and made it globally accessible to all classes. Adjusted how default options are constructed. [#GOATS-446]
• Refactored modal: Improved modal code for maintainability. [#GOATS-447]
• Refactored dragons app folder: Consolidated and organized code in the dragons app folder for better modularity and maintainability. [#GOATS-448]
• Refactored available recipes logic: Refactored the available recipes structure to simplify code and improve maintainability. Added a global event dispatcher to notify when a recipe is changed, allowing other components to react accordingly. [#GOATS-449]
• Refactored available files for observation type: Simplified the structure of available files by refactoring the code. Introduced helper functions to create unique IDs using observation type, observation class, and object name. [#GOATS-450]
• Refactored observation data organization: Enhanced how observation type, observation class, and object name organize recipes and files. Added a new endpoint to set up initial data for recipes and files for a specific run. [#GOATS-451]
• Refactored API grouping control: The API now allows users to specify fields to group for better DRAGONS use. [#GOATS-452]
• Refactored file identifiers in accordions: Refactored how files are displayed in accordions based on observation type, class, and object name. Introduced a helper class to manage these identifiers efficiently. [#GOATS-457]
• Refactored available files handling: Enhanced file filtering mechanisms and prepared for future expansion to include all files. Callbacks for filtering processes were integrated to ensure smooth operations. [#GOATS-458]
• Refactored recipe reduction. [#GOATS-459]
• General cleanup: Removed unnecessary data storage and added documentation. [#GOATS-461]
• Refactored WebSocket updates and app initialization. [#GOATS-462]

What's Changed

• GOATS-426: Add API to list DRAGONSRun output files. by @davner in #170
• GOATS-430: Connect API with UI for output file directory visibility. by @davner in #171
• GOATS-431: Add API backend for file management for DRAGONS runs. by @davner in #172
• GOATS-433: Link backend and frontend for DRAGONS output file operations. by @davner in #173
• GOATS-434: Implement uparms UI for recipe modification. by @davner in #174
• GOATS-435: Extend DRAGONS recipe "uparms" handling in API. by @davner in #175
• GOATS-436: Connect frontend to backend for uparms in DRAGONS reduction. by @davner in #176
• GOATS-437: Refactor DRAGONS logger for efficiency and cleanup. by @davner in #177
• GOATS-438: Refactor progress bar for recipes. by @davner in #178
• GOATS-439: Fix versioning issues for tomtoolkit, GOA, and astroquery. by @davner in #179
• GOATS-441: Refactor run panel and add loading animation. by @davner in #180
• GOATS-442: Refactor files table to improve group display and file toggling. by @davner in #181
• GOATS-446: Move DRAGONS API to singleton design and simplify interface. by @davner in #182
• GOATS-447: Refactor modal for maintainability and DRAGONS app integration. by @davner in #183
• GOATS-448: Refactor folder structure for dragons app code. by @davner in #184
• GOATS-449: Refactor available recipes and add global event dispatcher. by @davner in #185
• GOATS-450: Refactor available files for observation type. by @davner in #186
• GOATS-452: Enhance API grouping control for DRAGONS recipes and files. by @davner in #187
• GOATS-451: Refactor data organization and add new endpoint. by @davner in #188
• GOATS-457: Refactor file identifiers in accordions. by @davner in #189
• GOATS-458: Refactor available files filtering. by @davner in #190
• GOATS-459: Refactor recipe reduction. by @davner in #191
• GOATS-461: Remove unneeded data storage and add documentation. by @davner in #192
• GOATS-462: Refactor WebSocket updates and app initialization. by @davner in #193
• GOATS-412: Major refactor of DRAGONS app. by @davner in #194

Full Changelog: 24.9.0...24.10.0

24.9.0

GOATS 24.9.0 (2024-09-20)

New Features

• Enabled extended downloading from GOA: Added capability to download and link missing data from other observation IDs or calibration files. Users can now use standard stars, BPMs, and other resources from other observation IDs for use in DRAGONS reduction.​ [#GOATS-267]
• Updated file UI interactions: Connected UI components and API fetch functionalities to update, filter, group, and query available files for DRAGONS reductions. [#GOATS-379]
• Added date and time filtering: Enhanced DRAGONS file filtering by adding support for date, time, and datetime descriptors. Comprehensive tests were implemented for the new astrodata descriptor filtering features. [#GOATS-391]
• Refreshed dropdown on selection: Added a handler to clear the input text and refresh available options whenever a user selects an item from the multiselect dropdown for descriptor groups. [#GOATS-394]
• Included file count for 'All': Displayed the number of files when filtering to reduce confusion between filtering only and grouping with filtering. [#GOATS-396]
• Extended background worker timeout and made configurable: Allowed users to configure the time limit for background tasks via Django settings, enabling better control over task execution duration. [#GOATS-400]
• Added truncation for grouped values: Grouping values are now truncated to include file counts. [#GOATS-405]
• Enhanced UI with informational tooltips: Added clickable icons to the DRAGONS frontend that display tooltips explaining strict filtering options and available logical operators for filter expressions. [#GOATS-409]
• Added select-all/deselect-all functionality for files for observation types. [#GOATS-410]
• Design UI for calibration database: Completed the UI design and development for the calibration database. [#GOATS-415]
• Added file management capabilities to the calibration database: Users can now add files to, remove files from, and list files in the calibration database directly via the API. [#GOATS-417]
• Connected frontend with backend API for file removal and refresh: Integrated the frontend user interface with the backend API to enable file removal from the calibration database. Added a refresh button to update the database view. [#GOATS-420]
• Added file upload support: Enabled uploading files to the calibration database for DRAGONS reduction. [#GOATS-421]
• Developed output files UI: Developed a user interface container to manage and display output files for a DRAGONS reduction. [#GOATS-425]
• Enhanced file upload feedback and usability: Added a new column in the user interface to indicate which files were uploaded by users. Fixed an issue that prevented the re-upload of the same file consecutively. [#GOATS-427]

Changes

• Improved error handling for GOA downloads: Added error handling for file downloads with updates to the webpage's progress bar to reflect errors. Errors are now logged within the download model, providing users with detailed error messages when issues occur.​ [#GOATS-312]
• Sanitized run IDs for folder names: When a user provides a run ID for DRAGONS reduction, all characters unsuitable for a folder directory name are removed, and spaces are replaced with underscores. [#GOATS-337]
• Removed old bias filtering: Replaced with a more powerful file filtering system. [#GOATS-399]
• Enhanced product ID uniqueness: Made the product ID for a dataproduct more robust to fix integrity issues when adding the same dataproduct under different observations and targets. [#GOATS-401]
• Refactored run table classes for clarity and improve the maintainability of the DRAGONS UI. [#GOATS-413]

Bug Fixes

• Removed limit on multiselect dropdown options: The maximum number of options displayed in the multiselect dropdown has been removed, allowing for unrestricted selection from all available options. [#GOATS-390]
• Updated database model for DRAGONS runs: Corrected the database model to handle unique recipes per observation type and object name when the observation type is an object, addressing crashes for observation records with similar recipe requirements. [#GOATS-392]
• Fixed dataset referencing in DRAGONS interface: The observation record ID dataset attached to the DRAGONS interface was referenced improperly and has been corrected. [#GOATS-393]

What's Changed

• GOATS-379: Connect UI and API for file interactions. by @davner in #144
• GOATS-337: Sanitize run IDs for directory naming. by @davner in #145
• GOATS-312: Improved error handling for GOA downloads. by @davner in #146
• GOATS-392 GOATS-393: Fix dataset referencing and update database model. by @davner in #147
• GOATS-390: Remove limit for displaying options in multiselect dropdown. by @davner in #148
• GOATS-394: Add handler to clear and refresh options. by @davner in #149
• GOATS-391: Add date and time filtering support for DRAGONS files. by @davner in #150
• GOATS-400: Extend background worker timeout to 24 hours. by @davner in #151
• GOATS-399: Remove old bias filtering system. by @davner in #152
• GOATS-396: Display file count for 'All' group. by @davner in #153
• GOATS-401: Improve product ID uniqueness for dataproducts. by @davner in #154
• GOATS-267: Enhance data linking and downloading from GOA. by @davner in #155
• GOATS-405: Add truncation for grouping values. by @davner in #156
• GOATS-409: Enhance UI with informational tooltips. by @davner in #157
• GOATS-410: Implement select-all/deselect-all for files. by @davner in #158
• GOATS-413: Refactor run table for better maintainability. by @davner in #159
• GOATS-416: Bugfix for missing run ID for grouping and filtering. by @davner in #160
• GOATS-415: Design and develop UI for the calibration database. by @davner in #161
• GOATS-418: Add file IDs back to checkboxes. by @davner in #162
• GOATS-419: Select files in tbody, not runTable. by @davner in #163
• GOATS-417: Add file management capabilities to the calibration database. by @davner in #164
• GOATS-420: Connect frontend with backend for file management. by @davner in #165
• GOATS-421: Added file upload support for calibration database. by @davner in #166
• GOATS-422: Catch timeout for requests. by @davner in #167
• GOATS-425: Develop output files UI for DRAGONS reduction. by @davner in #168
• GOATS-427: Enhance user upload tracking in calibration database. by @davner in #169

Full Changelog: 24.8.0...24.9.0

24.8.0

GOATS 24.8.0 (2024-08-22)

New Features

• Added run information panel on DRAGONS UI: Displayed selected run details, including creation date, DRAGONS version, and output directory path. [#GOATS-332]
• Added UI components for file grouping and filtering: Introduced user interface elements that allow grouping and filtering of files, featuring a multiselect dropdown for selecting astrodata descriptors. [#GOATS-376]
• Enhanced file grouping and filtering: Added functionality to fetch files from the frontend to the grouping and filtering API backend. Implemented listeners for button clicks to query API from the form. [#GOATS-377]
• Added API endpoint for groups retrieval: Provided astrodata descriptors (groups) via API for DRAGONS runs and files. [#GOATS-378]
• Grouped files by astrodata descriptors: Implemented an API backend to group files by their astrodata descriptors and count the files accordingly. [#GOATS-380]
• Filtered files by astrodata descriptor values: Created an API backend to filter files based on expressions matching astrodata descriptor values. [#GOATS-381]

Changes

• Overhaul recipe assignment logic: Abandoned reliance on observation types for assigning recipes. Transitioned to using recipes modules, instruments, and tags to manage file recipes. This change enables GOATS to efficiently segregate files by their respective recipes and further distinguish different objects that may require unique recipes. The update prepares GOATS for integrating new instruments. [#GOATS-360]
• Extended help page for interactive mode: Enhanced help documentation to show how to enable interactive mode for specific primitives. Interactive mode is no longer the default setting. [#GOATS-367]

Bug Fixes

• Fixed modal and toast closing issues: Resolved a bug caused by the transition to Bootstrap 5. [#GOATS-356]
• Fixed help page docstring retrieval: Corrected an issue where docstrings were not properly fetched for the help page. Added tests to prevent in future. [#GOATS-371]

PRs

• GOATS-360: Overhaul recipe assignment logic. by @davner in #135
• GOATS-367: Update help documentation for interactive mode. by @davner in #136
• GOATS-371: Fix docstring retrieval on help page. by @davner in #137
• GOATS-356: Fix modal and toast closing with Bootstrap 5. by @davner in #138
• GOATS-332: Add run information panel on DRAGONS UI. by @davner in #139
• GOATS-378: Add API endpoint for groups retrieval. by @davner in #140
• GOATS-380 GOATS-381: Enhance file filtering and grouping capabilities. by @davner in #141
• GOATS-376: Add UI components for file grouping and filtering. by @davner in #142
• GOATS-377: Enhance file fetching and interaction. by @davner in #143

Full Changelog: 24.7.0...24.8.0

24.7.0

GOATS 24.7.0 (2024-07-23)

New Features

• Add Chrome extension link: Users can now click to access the Chrome extension store for installing antares2goats to enhance their GOATS experience from the ANTARES broker page. [#GOATS-294]
• Editing, resetting, and saving DRAGONS recipes: DRAGONS recipes now support editing, saving, and resetting to original states. Users can customize recipes during data reduction processes. [#GOATS-321]
• Enabled custom recipe input for DRAGONS: Users can now specify and utilize their own recipes in the DRAGONS reduction process. [#GOATS-345]
• Added UI for DRAGONS reduction help pages: Side offcanvas with animation opens and closes to display helpful information for users on click. [#GOATS-346]
• Added query parameter for detailed docs for primitives in API: Extended the DRAGONS files and recipes system to include a new query parameter. This parameter allows API responses to provide detailed documentation and descriptions of primitives used in a recipe. [#GOATS-349]
• Connected frontend and backend for help docs: Established linkage between the frontend and backend systems for fetching and displaying help documentation related to primitives. Designed the user interface to comprehensively present all components of numpy doc strings and parameters when available. [#GOATS-350]
• Implemented version-based recipe creation: Prevented redundant recipe entries in DRAGONS by creating base recipes only when the version changes. [#GOATS-358]
• Updated UI recipe selection: Added functionality to choose and display recipes dynamically in DRAGONS recipe cards. Enhanced user interface elements include ordered observation types and updated card titles. [#GOATS-359]

Changes

• Output directory now matches run ID: Removed unused setup form and refresh button for DRAGONS runs. Disabled the delete option but retained it as a placeholder. [#GOATS-305]
• Refactored UI for recipe management: Redesigned the user interface for managing observation type recipes and reductions. Now, only one reduction is displayed at a time, requiring users to toggle between them. This change simplifies the interface, helping users focus on one task at a time and reducing information overload. [#GOATS-351]
• Improved "Help" bar output: Preserved spacing in docstrings for improved readability and changed applied styles. [#GOATS-352]

Bug Fixes

• Fixed custom media directory issue: Resolved path handling for custom media directories when running DRAGONS and saving products. [#GOATS-304]
• Disabled automatic retries for failed DRAGONS reductions and GOA downloads. [#GOATS-335]
• Resolved bug for trying to set state of null element in UI. [#GOATS-340]
• Improved error handling for GOA timeouts when querying data products. [#GOATS-344]

Enhancements

• Enhanced GOATS startup and shutdown: Removed threading and implemented subprocesses. GOATS now exits cleanly, allowing sufficient time for all processes to shutdown properly. [#GOATS-336]
• Reduced file operations in DRAGONS recipe queries. [#GOATS-357]

Pull Requests

• GOATS-294: Add Chrome extension link to ANTARES broker page. by @davner in #119
• GOATS-335: Disable retries for dramatiq background tasks. by @davner in #120
• GOATS-304: Fix issue with custom media directory and DRAGONS. by @davner in #121
• GOATS-305: Sync output directory with run ID. by @davner in #122
• GOATS-340: Fixed bug trying to set state of a null element in UI. by @davner in #123
• GOATS-336: Enhance GOATS startup and shutdown. by @davner in #124
• GOATS-321: Support editing, resetting, and saving DRAGONS recipes. by @davner in #125
• GOATS-345: Enable custom recipe input for DRAGONS. by @davner in #126
• GOATS-346: Implement side offcanvas for help pages. by @davner in #127
• GOATS-344: Improve erorr handling when querying GOA. by @davner in #128
• GOATS-349: Add detailed docs query parameter to DRAGONS API. by @davner in #129
• GOATS-350: Connect frontend and backend for help documentation. by @davner in #130
• GOATS-351: Refactor UI for recipe management. by @davner in #131
• GOATS-357 GOATS-358: Reduce file operations in DRAGONS queries and implement version-based recipe creation. by @davner in #132
• GOATS-359: Update UI recipe selection. by @davner in #133
• GOATS-352: Improve "Help" bar output. by @davner in #134

Full Changelog: 24.6.0...24.7.0

24.6.0

GOATS 24.6.0 (2024-06-25)

New Features

• Extended pagination to include item count: Overrode bootstrap_pagination to show "Showing x-y of n" message. Updated HTML template to display item counts. [#GOATS-178]
• Implemented WebSocket support for DRAGONS logs: Developed a Channels consumer to handle real-time log messages from DRAGONS. Added a new WebSocket endpoint for DRAGONS updates and integrated a WebSocket logging handler. Expanded testing to cover Django Channels consumers. [#GOATS-286]
• Developed DRAGONS WebSocket logging: Developed a Python logging handler for WebSocket communication to provide real-time logs for the DRAGONS system. [#GOATS-290]
• Add backend for DRAGONS reduction: Developed an API to initiate and manage DRAGONS reduction processes in the background. Introduced a model to store details and updates of background tasks. Wrote comprehensive tests for the new backend infrastructure. [#GOATS-292]
• Enabled initiation of DRAGONS recipe reduction from the UI. [#GOATS-295]
• Added cancel endpoint for DRAGONS tasks: An API endpoint now allows canceling running or queued tasks in DRAGONS by setting the status of a recipe reduction to “canceled.” This action triggers the abortion of the background task. The update includes a new serializer to handle patches and extends tests to cover both valid and invalid patch scenarios. [#GOATS-299]
• Enabled running task cancellation from UI: Connected the frontend "Stop" button with the backend to enable users to cancel running tasks directly from the interface. Added logic to dynamically enable or disable "Start" and "Stop" buttons based on the current status of recipe reductions. [#GOATS-300]
• Display real-time logs on frontend with websocket: Built infrastructure to manage recipes for reduce runs, simplifying updates to specific recipes. Refactored recipe MVC. [#GOATS-301]
• Extended DRAGONS consumer for real-time recipe progress updates: Updated the UI to display initial progress information. Added utilities for easier real-time communication and refactored UI progress bars to lay the foundation for future enhancements. [#GOATS-302]
• Enabled interactive mode for select file types in recipe reduce: Integrated Bokeh for interactive visualization in ‘arc’, ‘flat’, and ‘object’ file types. [#GOATS-303]
• Wrote tests for additional Django Channels classes: Added unit tests for websocket classes responsible for the notification system. [#GOATS-307]
• Enhanced DRAGONS log autoscroll behavior: Updated logger to conditionally autoscroll based on the user's current scroll position. Methods intended for logger internal use were made private. [#GOATS-308]
• Cleared DRAGONS logs at recipe start. [#GOATS-309]
• Load running reductions on DRAGONS run select: Implemented synchronization of running reductions on page refresh or when a new run is selected. Added support for query parameters to fetch and limit reduction results in the API. [#GOATS-313]

Changes

• Update conda environment file and dependencies: Removed the set version for tomtoolkit. Fixed issue caused by the new version of tomtoolkit. [#GOATS-272]
• Added additional recipe reduce status feedback: Enhanced visibility of recipe reduce states and refined error handling in the DRAGONS reduce background task. [#GOATS-310]
• Enhanced recipe progress UI: Updated the progress bar to display different colors for different states and provide status label. [#GOATS-311]
• Switched to dramatiq for task management: GOATS now uses dramatiq for background tasks due to its support for aborting running tasks, a feature not available in huey. [#GOATS-315]

Bug Fixes

• Fixed websocket connection issue: Resolved a bug where websockets failed to open on the DRAGONS run page, restoring functionality for notifications and download progress updates. [#GOATS-314]