Cybersyn makes daily trading volumes & prices of all US equities/ETFs executed on the Nasdaq available in your Snowflake instance for free. Data is inclusive of pre-market/after hours activity and is released daily at 6:00am ET. Learn more in Cybersyn Docs:
I have been wondering how to support interactive / real-time web apps based on Snowflake data. I suppose pushing down to DuckDB a subset of data needed for a chart would be one way to do this...
If you’re pushing down the data, you’re losing the real-time capability no?
If you want fast, adhoc, real-time querying, load the data as it’s created directly into duckdb or clickhouse. Now you’ll have sub-100ms responses for most of your queries.
I'd assume they mean users interacting with the chart vs first load. So the user sees the base chart (Let's say 1MB of data on the server, less depending what gets pushed to the user) and then additional filters, aggregations, etc. are pretty cheap because the server has a local copy to query against
Our team at Cybersyn aggregated 300M+ domains in a single source. Domains are cleaned into a standardized format with any protocols and subdomains stripped away.
For a subset of domains, the dataset includes information on redirects such as a website’s redirect domain, the start/end dates for which the redirect relationship was observed, and whether or not a domain is the primary landing page. HTTP response statuses indicate whether a domain is active or inactive.
American Community Survey (ACS) 2018-2022 5-year estimates provide the most up-to-date social, economic, housing, and demographic statistics. Available down to zip code and census block groups, example topics include: median income, poverty rates, education attainment, mortgage status, gross rent, population, and home ownership rates
The raw data from the US Census Bureau is large and challenging to work with. Our team at Cybersyn made it easier to use by making the data available on Snowflake Marketplace in a consistent and intuitive form.
Yeah, our goal is to just make this data easily available in an easy-to-use tabular format in Snowflake. We occasionally visualize our datasets to demo them: https://app.cybersyn.com/home/ but haven't gotten to this one.
Thank you! I have found manipulable (complex cross tabs, time series etc) census and ACS data is really hard to access outside of those services you pay for
Yes, exactly! Sadly, there are entire commercial vendors dedicated to selling cleaned up versions of relatively simple cross-tabs. The ACS does extraordinary data surveying, but is less strong at structuring it, especially for analytical use cases.
If you are looking for future datasets to tackle, may I suggest ATUS or any HNWI-targeted data (not sure there is a public dataset of this last one tho)
Unlike the US Census released every 10 years, the American Community Survey is published annually. The data is difficult to wrangle but extremely important. Everything from income by zip-code, population growth by city, and home ownership statistics are published. There are even entire websites dedicated to just selling CSVs of income by zip-code.
Cybersyn made the data easier to work with by releasing it on Snowflake in a consistent, intuitive form.
It would be far easier if you had a simple JSON endpoint where I can pass a zip code and get the stats. That would be infinitely easier than figuring out things like how did I even use Snowflake?
Caden's amazing! We're trying to enable users to monetize in a different manner, beyond "selling their data off as a one-time profit opportunity"; moreso, we're gearing ourselves more towards allowing people to profitably drop data into AI companies in a way that minimizes initial risk for the startups, and then reap the rewards as they grow.
Really love the idea of being able to automatically toggle your data from specific companies. We can add some support to allow you to port your data to invest, and recommend startups with some analytics.
https://docs.cybersyn.com/getting-started/concepts/stock_pri...