SurvCancel: A Longitudinal Dataset and Benchmark for Dynamic Order Cancellation Prediction in On-Demand Ride-Sharing Systems

NeurIPS 2026Evaluations & Datasets Track

Huayang Liu1,* Mingyu Zheng1,* Wei Zhang1,2,† Yijun Bian3 Kai Wang1,2

1 School of Vehicle and Mobility, Tsinghua University

2 State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University

3 Department of Computer Science, University of Copenhagen

* Equal contribution† Corresponding author

Abstract

Passenger-order cancellation is a critical source of uncertainty in on-demand ride-sharing systems, where requests, assignments, and vehicle routes evolve continuously in real time. Due to shared capacity and coupled routes, a single cancellation may propagate through downstream matching and routing decisions, degrading system efficiency and service reliability. However, cancellation prediction in on-demand ride-sharing remains difficult to study systematically due to the lack of large-scale longitudinal datasets, standardized evaluation protocols, and fine-grained real-time benchmarks.

In this work, we introduce SurvCancel, a real-world longitudinal dataset and benchmark for dynamic passenger-order cancellation prediction in on-demand ride-sharing systems. Constructed from operational logs collected across four service regions, SurvCancel contains 173K released orders and 7.9M time-dependent snapshots, capturing both order-level attributes and the evolving supply-demand environment throughout each order's lifecycle. We formulate cancellation prediction as a stage-conditioned dynamic survival task: at each landmark time, models estimate short-term cancellation risk from the historical state sequence observed so far. Evaluating representative classical and neural survival models under both in-distribution and distribution-shift protocols reveals stage-dependent model behavior and distinct robustness patterns across regions and time periods.

Dataset Overview

The paper's SurvCancel overview: two pre-pickup lifecycle stages, possible cancellation events, history-only landmark prediction, and order-level and environment-level snapshot features.
Dynamic cancellation prediction across the order lifecycle. Each 10-second snapshot captures the focal order's status alongside its surrounding supply-demand context. Under our landmark prediction protocol [7], models observe only the prefix history up to the landmark time τ and estimate short-term cancellation risk within a stage-specific future window (60 s before matching; 30 s after matching).

The SurvCancel Dataset

173KOrders
7.9MSnapshots
4Service regions
10 sSnapshot interval

Each order is tracked longitudinally along its pre-pickup trajectory—from creation through vehicle matching to pickup or cancellation—jointly with nearby active requests and available fleet capacity. Alongside the raw variable-length snapshots, we release Clean58, a standardized 58-dimensional representation that provides a unified input interface across model families.

Our benchmark suite evaluates models under in-distribution, leave-one-region-out (LORO), and chronological splits using order-level partitioning and shared test landmarks.

Experiments

We benchmark six representative survival models alongside RiskPool, our lightweight history-pooling architecture. All methods are evaluated on the shared Clean58 representation and fixed test landmarks across three random seeds.

Evaluated baselines span static competing-risk survival modeling (DeepHit [1]), recurrent dynamic survival modeling (DynDeepHit [2]), Transformer-based survival models (SurvTRACE [3] and TraCeR [4]), Landmark Cox regression [5][7], and Random Survival Forests (RSF) [6].

Main Benchmark

We evaluate both risk discrimination via the cancel-window C-index [8] and probabilistic accuracy via the cancel-window Brier score [9].

Seven models are ranked separately for each metric. RiskPool has the highest overall cancel-window C-index, 0.7527. RSF has the lowest Brier score, 0.0757. Blue highlights the best mean in each panel. Points show means over three training seeds.
Risk ranking and probability calibration favor different model families. RiskPool achieves the best overall C-index (0.7527), whereas RSF [6] attains the lowest raw Brier score (0.0757), demonstrating that strong risk ordering does not automatically guarantee well-calibrated cancellation probabilities. Models are ordered from best to worst in each panel (mean over three seeds; best in blue).
View stage-wise benchmark scores
In-distribution test results on Clean58 (mean over three seeds). Stage 0: pre-matching (60 s horizon); Stage 1: post-matching (30 s horizon).
ModelOverall C-index ↑Stage 0 C-index ↑Stage 1 C-index ↑Brier ↓
RiskPool (ours)0.75270.76500.70610.0768
DynDeepHit [2]0.74960.76180.70900.0765
TraCeR [4]0.74850.76160.70000.0894
RSF [6]0.74580.75610.70020.0757
SurvTRACE [3]0.73500.76080.63270.0928
Landmark Cox [5][7]0.72350.73420.67410.0776
DeepHit [1]0.68270.71270.58660.0793

Generalization Across Regions and Time

We evaluate the four top-performing models under spatial transfer (leave-one-region-out across R1–R4) and temporal shift (training on earlier dates and testing on future orders), comparing each shifted C-index against its corresponding in-distribution (ID) reference.

All four models lose C-index under regional and temporal shifts. DynDeepHit leads the unseen-region comparison at 0.7245, a change of minus 0.0315 from its ID macro reference. RiskPool leads the later-date comparison at 0.7351, a change of minus 0.0176 from its pooled ID reference. Hollow gray points are ID scores; solid blue points are shifted scores.
Distinct robustness patterns emerge under spatial versus temporal shift. While all models experience moderate degradation relative to their ID references (hollow gray points), DynDeepHit [2] achieves the highest macro-average C-index on unseen regions (0.7245), whereas RiskPool leads under chronological evaluation on future orders (0.7351).

References

  1. [1]
    Changhee Lee, William R. Zame, Jinsung Yoon, and Mihaela van der Schaar. DeepHit: A Deep Learning Approach to Survival Analysis With Competing Risks. AAAI, 2018.
  2. [2]
    Changhee Lee, Jinsung Yoon, and Mihaela van der Schaar. Dynamic-DeepHit: A Deep Learning Approach for Dynamic Survival Analysis With Competing Risks Based on Longitudinal Data. IEEE Transactions on Biomedical Engineering, 67(1):122–133, 2020.
  3. [3]
    Zifeng Wang and Jimeng Sun. SurvTRACE: Transformers for Survival Analysis With Competing Events. ACM BCB, 2022.
  4. [4]
    Maxmillan Ries and Sohan Seth. TraCeR: Transformer-Based Competing Risk Analysis with Longitudinal Covariates. arXiv:2512.18129, 2025.
  5. [5]
    D. R. Cox. Regression Models and Life-Tables. Journal of the Royal Statistical Society, Series B, 34(2):187–220, 1972.
  6. [6]
    Hemant Ishwaran, Udaya B. Kogalur, Eugene H. Blackstone, and Michael S. Lauer. Random Survival Forests. The Annals of Applied Statistics, 2(3):841–860, 2008.
  7. [7]
    Hans C. van Houwelingen. Dynamic Prediction by Landmarking in Event History Analysis. Scandinavian Journal of Statistics, 34(1):70–85, 2007.
  8. [8]
    Frank E. Harrell Jr., Kerry L. Lee, Robert M. Califf, David B. Pryor, and Robert A. Rosati. Regression Modelling Strategies for Improved Prognostic Prediction. Statistics in Medicine, 3(2):143–152, 1984.
  9. [9]
    Glenn W. Brier. Verification of Forecasts Expressed in Terms of Probability. Monthly Weather Review, 78(1):1–3, 1950.

BibTeX

@inproceedings{liu2026survcancel,
  title     = {SurvCancel: A Longitudinal Dataset and Benchmark for Dynamic Order Cancellation Prediction in On-Demand Ride-Sharing Systems},
  author    = {Liu, Huayang and Zheng, Mingyu and Zhang, Wei and Bian, Yijun and Wang, Kai},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS), Evaluations and Datasets Track},
  year      = {2026}
}