Hackathon
Details

The ML task to be solved is a supervised learning one, and more particularly a multi-targets regression problem.

Features:

Features: Each training datapoint consists of a set of 55 noisy light curves (one per wavelength, each being a timeseries of 300 timesteps) and a set of 6 additional stellar and planetary parameters. All these are real numbers. For more details on what a light curve is and what we are modelling, go to the Science page.

Targets:

The goal is to predict a set of 55 real values (relative radii, one per wavelength) for any given datapoint (lightcurve).

Event Rules

1. Each team must have a unique alias.
2. The organisers will use the provided code, models and solutions only for the purposes of checking whether the contest rules (4) & (5) are not broken. The organisers will not use any result without the authors’ permission.
3. Participants must not use TauREx 3 or similar atmospheric retrieval codes to retrieve the test-set forward models. This would be considered test-set leakage and against the spirit of the competition.
4. Participants must not have access to the heldout testset's ground truth before the competition’s closing date. If they do, they will be disqualified.
5. For an entry to count as a winning entry, it must not rely heavily (as judged by the organisers) on hard-coded elements that are solely deemed to be due to test-set leakage.
6. The organisers reserve the right to interpret and enforce the above rules at their discretion to maintain a respectful and inclusive environment for all participants.

In the case of a draw

Should two or more top-ranked participants have the same final score, we will require the participants to submit their algorithms. This is to check that no plagiarism has occured.

Reasons for disqualification

In case of plagiarism, forced test set leakage, failure to produce a description of the solution when requested or to conform with the multiple accounts or the team submissions policies, a participant will be disqualified and the next-in-rank participant will be considered in their place.

To evaluate a new solution (model), the participants are requested to upload its predictions on the test dataset via the score calulcator page. See the Data Formats page for information on the upload format. A score will then be automatically calculated for the solution.

The score reported is based on the average of the absolute error per target (i.e. on the relative radii) across all test set examples \(i\) and all wavelengths \(j\) and is given by:

$$Score = 10^4 - \frac{\sum_{i \in Test} \sum_{j=1}^{55} w_{ij} 2 y_{ij} |\hat{y}_{ij} - y_{ij}|}{\sum_{i \in Test} \sum_{j=1}^{55} {w_{ij}}}10^6,$$

where $$y_{ij}$$ is the true relative radius and $$\hat{y}_{ij}$$ the predicted relative radius of the $$\(j\)-th$$ wavelength of the $$\(i\)-th$$ test set example. $$w_{ij}$$ is treated as uniform in this hackathon, meaning every wavelengths and test examples are treated as equally weighted.

The higher the score, the better your ranking. The maximum achievable score is 10000. The score is not lower-bounded (i.e. can be negative), but reasonable attempts (e.g. predicting the average target value for all test datapoints) should not produce scores below 4000.