My brief foray into SQL and data engineering through ArcGIS has me absolutely amazed at how seamless so much of the internet is. Even doing a spatial query of a small town can take several minutes for a dataset with say 30,000 records. I tried some of my usual geospatial join and query techniques on a dataset of 175 million and my timeframe quickly approached the heat death of the universe.
My brief foray into SQL and data engineering through ArcGIS has me absolutely amazed at how seamless so much of the internet is. Even doing a spatial query of a small town can take several minutes for a dataset with say 30,000 records. I tried some of my usual geospatial join and query techniques on a dataset of 175 million and my timeframe quickly approached the heat death of the universe.