Home   >  Competitions   > 

Evaluation

The data sets contain 3 million Mobike trips, each trip has a unique order id. Participants need to build their models on the training set and predict three most probable destinations of each trip. Please note that these predictions are ordered by their probability, from high to low.

 

Submission File

Please refer to sample_submission.csv for submission format. The submission files are in csv format without any header:

orderid, end_location1, end_location2, end_location3

with each line of the file corresponds to one trip order id (orderid), followed by 3 destination blocks (end_location). These tree predictions in one line shall be different from each other. Halfwidth comma shall be used between orderid and destination, or different destinations.

One example:

6875, w5shgyte, wx4dztn8, wx4dzmkq

4569, wx4er4e7, wx4g57u2, wx4dzmkq

 

In which, 6875 and 4569 are orderid, the other data is end_location.

 

Evaluation Metrics

Submissions are evaluated according to the Mean Average Precision @3 (MAP@3)

Where |U| is the number of check in events, P(k) is the precision at cutoff k, n is the number of predicted locations.

 

Here is a nice reference on MAP@3, with code:

https://github.com/benhamner/Metrics/blob/master/Python/ml_metrics/average_precision.py