Time-Domain Needles in Rubin's Haystacks

Harvard Center for Astrophysics, Cambridge, MA | April 17-19, 2024

Hackathon Projects

Brief description of hack: Rubin Observatory is slated to make a significant contribution to the study of the Solar System, delivering over a billion highly precise observations of millions of Solar System objects (5mmag photometry and 10mas astrometry, per observation, at the bright end).

Current estimates show yields ranging from ~100,000 new discoveries of nearby NEOs, to 5.5 million for the main belt, and ~40,000 for KBO populations. The majority of these objects will receive hundreds of observations in multiple bandpasses. This dataset presents tremendous opportunities for Solar System science. Cometary activity in the Solar System is typically detected by looking for extended point spread functions (PSFs), but when ten years of photometric monitoring Rubin Observatory opens a new window into the transient Solar System.

In this project, I'm interested in exploring what techniques and algorithms can be applied to find activity in distant Solar System objects through photometry alone via jumps or unexpected changes in brightness.

Assigned participants: Tri Nguyen, Cecilia Garraffo
Slack channel for the hack in hackathon workspace: #cometary-activity
Github Repo: https://github.com/mschwamb/rubin_haystack_solarsystem

Brief description of hack: Several successful classifiers have been developed in preparation for the Vera Rubin Observatory. However, anomaly detection has eluded much of the literature. In this hack, we will repurpose existing classifiers for anomaly detection.

One key difficulty in developing anomaly detectors for astronomical transients is to identify a feature space that separates anomalies well. Most attempts have relied either on hand-crafted features extracted from light curves or on features generated through unsupervised representation learning. In this hack, we will use the penultimate layer of a neural network classifier as the latent space for anomaly detection. We will then use “Multi-Class Isolation Forests (MCIF)” to identify anomalies in this latent space.

This method was developed in a recent paper and applied to ZTF simulations using a recurrent neural network classifier.
Participants can take this hack in multiple directions:

1. Gather a different dataset.
2. Gather a real dataset of ZTF (or PanSTARRs) labelled transients and apply the same methodology in the notebook.
3. Use a different classifier as the latent space for MCIF anomaly detection
This approach is not limited to transients, and any other dataset with labelled classes can be used with the provided anomaly detection framework.

Assigned participants: Jun Yang, Nabeel Rehemtulla, Andrés Plazas Malagón, Kaylee de Soto, Ved Shah
Slack channel for the hack in hackathon workspace: #transient-classification
Github Repo: https://github.com/Rithwik-G/AstroMCAD

Brief description of hack: As supernova discovery rates grow beyond exponential scaling, strategies for science-specific follow-up will soon rely on more than classification alone. Soon, strategies will be designed to capture specific sub-types of popular classes (e.g., an SN Ia in an elliptical galaxy; a nearby SN IIn with a secondary bump and a compact host galaxy). A scalable transient similarity search algorithm is an essential tool for these programs, and for searching for specific anomalous transients among billions. This is the motivation underlying the Light Curve AI Similarity Search (LAISS), which uses parametric features of supernova light curves and host galaxies to flag anomalies and conduct targeted transient queries.

LAISS works well finding anomalous long-duration transients, but its approach has not been well tested in the limit of sparse and incomplete light curves. In this hack, we will:

1. Evaluate the performance of similarity searches in these regimes
2. Push to improve both its speed and the relevance of its retrieved events.
3. Explore similarity searches with other classes of transients (e.g., AGN, TDEs)

Assigned participants: Gautham Narayan, Maryam Hussaini, Conor Ransome, Wasundara Athukoralalage
Slack channel for the hack in hackathon workspace: #laiss-realtime
Github Repo: https://github.com/alexandergagliano/laiss_timedomainanomalies

Brief description of hack:Persistent homology is the flagship technique of TDA: here we consider applying it to time series to identify outliers (maybe) via learning the (non-Hilbert) manifold that persistence diagrams exist in and then looking for off-manifold points.
Assigned participants: Carolina Cuesta-Lazaro, Franc O, Yvette Cendes
Slack channel for the hack in hackathon workspace: #topological-data-analysis
Github Repo: https://github.com/doccosmos/pershout

Brief description of hack: Details to come!
Assigned participants: Siddharth Chaini, Shar Daniels, Riley Clarke, Fiorenzo Stoppa
Slack channel for the hack in hackathon workspace: #neural-autoencoder-alerts

Brief description of hack: Light echoes (LE) are the reflection of stellar explosions on interstellar dust. LE reaches Earth with a time delay, can be even centuries, providing us with a unique opportunity to study historical transients, such as supernovae (SNe). By comparing the spectra obtained from LE with known SNe spectra, it is possible to classify the type of source responsible for generating these LEs.

Our science goal is to build a LE spectra library and develop an algorithm to identify the type of sources of LEs. We will use the spectra and photometry data from Open Supernova Catalog and spectral observations from three LEs in the Large Magellanic Cloud (LMC).
Assigned participants: Caroline Huang, Daichi Hiramatsu
Slack channel for the hack in hackathon workspace: #supernova-remnant-spectroscopy
Github Repo: https://github.com/xiaolng/LESID/

Brief description of hack: Microlensing happens when light coming from a distant source star (for instance near the Galactic Bulge) is lensed by the gravitational field of an object (a star, a rogue planet, a black hole, a stellar binary system, a planetary system, etc) closer to the observer along the line of sight to the source star. In the absence of second order effects like microlensing parallax, finite source effect, or multiple lenses, the light curve will be a symmetric single lens microlensing curve. The second-order effects will cause anomalies on this curve. Early detection of the anomalous events will lead to better allocating follow-up resources and capturing the details of the curve to better model that.

Classification of microlensing events is already implemented in at least one of the LSST brokers. This project aims at investigating how well anomalous microlensing events can be flagged to be prioritized for follow-up.
Assigned participants: Alex Malz
Slack channel for the hack in hackathon workspace: #microlensing-elasticc
Github Repo: https://github.com/Somayeh91/CfA_hackathon_anomalous_microlensing

Brief description of hack: Periodic signatures in time-domain observations of quasars have been used to search for binary supermassive black holes (SMBHs). These searches, across existing time-domain surveys, have produced several hundred candidates. The general stochastic variability of quasars, however, can masquerade as a false-positive periodic signal, especially when monitoring cadence and duration are limited. As Rubin will observe millions of quasars, it will also open a new frontier for electromagnetic detection of binary SMBHs.

In this Hack, we will explore the application of basic machine learning techniques to the binary detection problem by using thousands of synthetic Rubin observations of both binary and single quasars. The goals are as follows:

1. Identify pre-processing needs for light curve data.
2. Apply out-of-the-box, open-source ML classification algorithms from popular Python packages, such as sci-kit learn and keras.
3. Identify common pitfalls, analysis missteps, and general recommendations for applying this detection method to this type of data.
4. Stretch goal: explore need for and generate more complex, homebrewed algorithms.

Assigned participants:
Slack channel for the hack in hackathon workspace: #binary-smbhs
Github Repo: https://github.com/megcdavis/RubinBinariesHackathon