Task

Participants will collaborate in teams (4 - 5 people) to devise machine learning-driven solutions aimed at converting noisy observations (referred to as lightcurves) into transmission spectra. The objective is to approximate the ground truth spectra as closely as achievable. The winning team(s) will be announced as the victor(s) of the hackathon and will receive a prize.

Evaluation

Your solution will be judged on 2 criteria: 1. Its performance on the final heldout datasets and 2. the interpretability of the model. A panel of judges comprising of experts in Astrophysics and Machine Learning will access and score your solution according to how interpretable they are. To ensure the models are comparable among teams, participants will work on a fixed metric(See Scoring Metrics tab) that measures the distance between ground truth and the prediction. The best overall model in both criteria will be declared winner of this hackathon.

Model Restrictions

There is no restriction on the models, algorithms or data preprocessing techniques, neither on the programming languages, environments or tools used for your implementation. You are also free to use data augmentation techniques, pretrained models or any prior domain knowledge not included in the provided dataset. Finally, you are free to choose your own way of splitting the training data between training and validation sets and to use as many of the provided datapoints or features as you wish – or can handle

The ML task to be solved is a supervised learning one, and more particularly a multi-targets regression problem.

Features:

Features: Each training datapoint consists of a set of 55 noisy light curves (one per wavelength, each being a timeseries of 300 timesteps) and a set of 6 additional stellar and planetary parameters. All these are real numbers. For more details on what a light curve is and what we are modelling, go to the Science page.

Targets:

The goal is to predict a set of 55 real values (relative radii, one per wavelength) for any given datapoint (lightcurve).

Event Rules

1. Each team must have a unique alias.
2. Team names containing any form of offensive language, including but not limited to profanity, derogatory terms, hate speech, or discriminatory language based on race, gender, religion, sexual orientation, or any other characteristic, are strictly prohibited. Any team found to be in violation of this rule will be disqualified from participation in the event, and further disciplinary action may be taken as deemed appropriate by the organisers.
3. The organisers will use the provided code, models and solutions only for the purposes of checking whether the contest rules (5) & (6) are not broken. The organisers will not use any result without the authors’ permission.
4. Participants must not use TauREx 3 or similar atmospheric retrieval codes to retrieve the test-set forward models. This would be considered test-set leakage and against the spirit of the competition.
5. Participants must not have access to the heldout testset's ground truth before the competition’s closing date. If they do, they will be disqualified.
6. For an entry to count as a winning entry, it must not rely heavily (as judged by the organisers) on hard-coded elements that are solely deemed to be due to test-set leakage.
7. The organisers reserve the right to interpret and enforce the above rules at their discretion to maintain a respectful and inclusive environment for all participants.

In the case of a draw

Should two or more top-ranked participants have the same final score, we will require the participants to submit their algorithms. This is to check that no plagiarism has occured.

Reasons for disqualification

In case of plagiarism, forced test set leakage, failure to produce a description of the solution when requested or to conform with the multiple accounts or the team submissions policies, a participant will be disqualified and the next-in-rank participant will be considered in their place.

To evaluate a new solution (model), the participants are requested to upload its predictions on the test dataset via the score calulcator page. See the Data Formats page for information on the upload format. A score will then be automatically calculated for the solution.

The score reported is based on the average of the absolute error per target (i.e. on the relative radii) across all test set examples \(i\) and all wavelengths \(j\) and is given by:

Score = 10^4 - \frac{\sum_{i \in Test} \sum_{j=1}^{55} w_{ij} 2 y_{ij} |\hat{y}_{ij} - y_{ij}|}{\sum_{i \in Test} \sum_{j=1}^{55} {w_{ij}}}10^6,



where y_{ij} is the true relative radius and \hat{y}_{ij} the predicted relative radius of the \(j\)-th wavelength of the \(i\)-th test set example. w_{ij} is treated as uniform in this hackathon, meaning every wavelengths and test examples are treated as equally weighted.

The higher the score, the better your ranking. The maximum achievable score is 10000. The score is not lower-bounded (i.e. can be negative), but reasonable attempts (e.g. predicting the average target value for all test datapoints) should not produce scores below 4000.
https://colab.research.google.com/drive/18t-a781ZDPCg_E_mATPrawSkg9M7fOWn?usp=sharing