My AI model hit 92% accuracy on a test set I thought it would bomb

I was training a small image classifier on my own dataset of local bird photos, maybe 500 total. I figured it would top out at 70% at best because my pictures aren't perfect. Ran the final test this morning and it scored 92%. The key was using a simple data augmentation script I found on GitHub called 'ImgAug Lite' to rotate and flip my training images. Totally changed my view on what's possible with limited data. Has anyone else had a small project perform way better than you expected?

3 comments

Log in to join the discussion

Log In

3 Comments

gracem326mo ago

Honestly that's a huge red flag for overfitting. 92% on 500 images? Your test set is probably way too similar to your training data now.

7

hill.mila6mo ago

Maybe they used the same source for both sets.

8

wendy_carr5mo ago

Seriously, did you check for data leakage? @gracem32 is right about overfitting, but your test set could be tainted. You need to verify the split was clean before celebrating.

7