Been doing some propensity modelling recently using R glm package, but upon recommendation from a professional in my network, I wanted to try k-fold cross validation to asses the accuracy of my model.
So doing some research I found a great article on fitting models on this site (which I have linked to before):
This is a really great site – such good clear examples of R code, with comments, so you can understand what is going on. The models in this example is a linear model, while I am using a glm, so instead of using cv.lm, I can use cv.glm to run the cross validation.
# BINOMIAL REGRESSION WITH 10 FOLD Cross Validation
e_data = read.csv("G:/Binomial Regression/set.csv", sep=',', header= TRUE, stringsAsFactors = FALSE)
model <- glm(Y ~ X1 + X2 + X3, family=binomial(link = "logit"))
cv.err <- cv.glm(lease_data, model, K=10) # 10 fold cross-validation
The deltas should be compared across different models, and indeed against an MSE you may have calculated using a more tradition test/validation set.
NB: here as a good article on cross validation by Rob Hyndman : http://robjhyndman.com/researchtips/crossvalidation/