'Comically bad' datasets used to train clinical models for stroke and diabetes

Legend2440•1 day ago

A lot of researchers think their job is to build models. They don't want to collect their own data, so they go find whatever dataset they can on kaggle or from a previous paper or wherever.

This is backwards. The model is the easy part. Getting good data is 99% of the job, and nearly any clown can make a good model once you hand them a good dataset.

skvmb•1 day ago

As a clown, I can confirm.

If you hand me a clean, well-labeled, representative dataset, I can make the model do a respectable little dance by lunch.

If you hand me a Kaggle CSV with duplicated rows, target leakage, mislabeled outcomes, and columns named final_final_v2_REAL, suddenly I’m not doing ML anymore. I’m doing archaeology with a red nose on.

The model is the balloon animal. The dataset is the elephant you had to drag into the tent.

steve_adams_86•1 day ago

This holds in software as well. I encounter people trying to build solutions for problems that might not even exist, even in the context of addressing a specific bug. The act of measuring and collecting data is hard work, pretty boring sometimes, and often prescriptive in ways that aren't appealing. It's like we'd rather guess and use the ambiguity to allow ourselves to explore solutions we're more interested in. The alternative is manually profiling and poring through logs, so, I kind of get it.

nradov•1 day ago

For a lot of clinical decision support use cases you don't even need fancy AI models to get accurate results. If you have good quality cleansed data you can literally just import it into Excel and run a simple linear regression analysis. But unfortunately that won't get you a reputation as an "AI thought leader".

kenjackson•1 day ago

Actually a simple flow-chart works for a large number of use cases. That said, there are a lot of use cases where we don't have a simple way to run a linear regression model to get reasonable results where "AI" does seem to work well.

QuercusMax•1 day ago

You just need to figure out a way to brand that as a new, resource-conserving AI model.

actionfromafar•1 day ago

I think it needs a cool name.

i7l•about 20 hours ago

So true and it's been like that for ages. It's why I called these people rogue data scientists five years ago:

https://ianreppel.org/how-to-spot-a-rogue-data-scientist/

matusp•1 day ago

Dataset quality is a huge issue in ML in general. You can often list a few dozen random samples from any given dataset and you will find out something weird going on instantly.

msarrel•about 5 hours ago

Old problems are new again.

'Comically bad' datasets used to train clinical models for stroke and diabetes

⚡ Community Insights

Discussion (11 Comments)Read Original on HackerNews