ML Monitoring interview questions

1. A.a change in the relationship between inputs and the target, degrading model accuracy over time
2. B.a central system that serves consistent, versioned features to both training and online inference
3. C.a change in the input feature distribution from what the model was trained on
4. D.the delay before true labels arrive, which forces proxy metrics to detect degradation meanwhile

Answer + AI explanation with Pro

1. A.Population Stability Index (PSI)
2. B.Training-serving skew
3. C.Concept drift
4. D.Data drift

Answer + AI explanation with Pro

1. A.Data drift — a change in the input feature distribution from what the model was trained on
2. B.Data drift — tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
3. C.Data drift — a mismatch between features computed in training and those at serving time, hurting live accuracy
4. D.Data drift — a metric quantifying how much a feature or score distribution has shifted between two periods

Answer + AI explanation with Pro

1. A.a metric quantifying how much a feature or score distribution has shifted between two periods
2. B.the delay before true labels arrive, which forces proxy metrics to detect degradation meanwhile
3. C.a change in the input feature distribution from what the model was trained on
4. D.a change in the relationship between inputs and the target, degrading model accuracy over time

Answer + AI explanation with Pro

1. A.Concept drift
2. B.Population Stability Index (PSI)
3. C.Ground-truth lag
4. D.Model performance monitoring

Answer + AI explanation with Pro

1. A.Concept drift — tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
2. B.Concept drift — a central system that serves consistent, versioned features to both training and online inference
3. C.Concept drift — a metric quantifying how much a feature or score distribution has shifted between two periods
4. D.Concept drift — a change in the relationship between inputs and the target, degrading model accuracy over time

Answer + AI explanation with Pro

1. A.a metric quantifying how much a feature or score distribution has shifted between two periods
2. B.a change in the input feature distribution from what the model was trained on
3. C.a mismatch between features computed in training and those at serving time, hurting live accuracy
4. D.a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.Population Stability Index (PSI)
2. B.Ground-truth lag
3. C.Model performance monitoring
4. D.Training-serving skew

Answer + AI explanation with Pro

1. A.Training-serving skew — a mismatch between features computed in training and those at serving time, hurting live accuracy
2. B.Training-serving skew — a metric quantifying how much a feature or score distribution has shifted between two periods
3. C.Training-serving skew — a change in the input feature distribution from what the model was trained on
4. D.Training-serving skew — a change in the relationship between inputs and the target, degrading model accuracy over time

Answer + AI explanation with Pro

1. A.tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
2. B.a change in the input feature distribution from what the model was trained on
3. C.a change in the relationship between inputs and the target, degrading model accuracy over time
4. D.a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.Concept drift
2. B.Feature store
3. C.Data drift
4. D.Training-serving skew

Answer + AI explanation with Pro

1. A.Feature store — a change in the input feature distribution from what the model was trained on
2. B.Feature store — a mismatch between features computed in training and those at serving time, hurting live accuracy
3. C.Feature store — a change in the relationship between inputs and the target, degrading model accuracy over time
4. D.Feature store — a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
2. B.a metric quantifying how much a feature or score distribution has shifted between two periods
3. C.the delay before true labels arrive, which forces proxy metrics to detect degradation meanwhile
4. D.a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.Feature store
2. B.Population Stability Index (PSI)
3. C.Model performance monitoring
4. D.Data drift

Answer + AI explanation with Pro

1. A.Population Stability Index (PSI) — tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
2. B.Population Stability Index (PSI) — a metric quantifying how much a feature or score distribution has shifted between two periods
3. C.Population Stability Index (PSI) — a change in the input feature distribution from what the model was trained on
4. D.Population Stability Index (PSI) — a mismatch between features computed in training and those at serving time, hurting live accuracy

Answer + AI explanation with Pro

1. A.the delay before true labels arrive, which forces proxy metrics to detect degradation meanwhile
2. B.a mismatch between features computed in training and those at serving time, hurting live accuracy
3. C.a metric quantifying how much a feature or score distribution has shifted between two periods
4. D.a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.Concept drift
2. B.Ground-truth lag
3. C.Population Stability Index (PSI)
4. D.Training-serving skew

Answer + AI explanation with Pro

1. A.Ground-truth lag — the delay before true labels arrive, which forces proxy metrics to detect degradation meanwhile
2. B.Ground-truth lag — tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
3. C.Ground-truth lag — a metric quantifying how much a feature or score distribution has shifted between two periods
4. D.Ground-truth lag — a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.a mismatch between features computed in training and those at serving time, hurting live accuracy
2. B.tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
3. C.a change in the input feature distribution from what the model was trained on
4. D.a central system that serves consistent, versioned features to both training and online inference

Answer + AI explanation with Pro

1. A.Model performance monitoring
2. B.Feature store
3. C.Population Stability Index (PSI)
4. D.Ground-truth lag

Answer + AI explanation with Pro

1. A.Model performance monitoring — a central system that serves consistent, versioned features to both training and online inference
2. B.Model performance monitoring — tracking live accuracy, precision/recall or business KPIs to trigger retraining or rollback
3. C.Model performance monitoring — a mismatch between features computed in training and those at serving time, hurting live accuracy
4. D.Model performance monitoring — a change in the input feature distribution from what the model was trained on

Answer + AI explanation with Pro