Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

How do you infer your Parquet schemas?


You infer the types of the source data.

For example you can go through say 1% of your data and for each column see if you can coerce all of the values to a float, int, date, string etc. And then from there you can set the Parquet schema with proper types.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: