Look at this typical forecast (thanks Google).
If I go hiking, will the trails be muddy? Do my plants need water today? Does my flat roof have puddles on it? All questions I can’t answer because apps never seem to show recent weather history. I decided to build a weather app on top of the SingleStore Free Shared Tier to answer my own questions. Disclaimer — I work for SingleStore — so make up your own mind about whether this is a “when all you have is a hammer” type of situation 😉.
TL;DR - weather.bingo
Now look at this beautiful view of Seattle. The trails are not dry. The plants do not need water. And the roof once again has a puddle. Read on for a short explanation of the services used. I’ll write more about the architecture and implementation on some follow-up posts.
Data Source
A weather app is data-driven; so I needed a source. In a methodical approach I searched “best weather APIs 2024” and found this assuredly biased blog post. I evaluated a few from the list and ended up with visualcrossing.com which has a single easy API for both forecast and historical data, a free tier, and a low-cost flat-fee paid tier (for personal stuff I try to avoid pay-per-call to avoid the risk of accidental overages).
Data Cache
I wanted a data cache for a few reasons. First by by selecting and managing my own data cache, I have more direct control over the user experience of the app. I can pre-fetch data when appropriate to reduce latency, and if this app becomes popular (narrator: it won’t) there are clear options to scale up. Second, using a cache helps me control my usage of the Visual Crossing API. This API charges units called “records” for each request. A 15-day forecast is one record, but for historical data the cost goes up - one record for each individual historical hour or day. Since historical data (probably) doesn’t change much, it’s a waste of records to retrieve it multiple times. While other consumers of Visual Crossing surely always want the very latest uncached information, for my app’s requirements I’ve decided that I’ll cache forecast data for one hour and historical data for an extended period.
Redis is commonly used as a data cache, but in this case I felt SingleStore was a better choice due to the way it compresses JSON/BSON data, and also the new shared free tier option.
SingleStore and JSON/BSON
SingleStore stores JSON and BSON by default in a columnar storage format. Each path in the document stored in a separate column, using a similar approach to Parquet. This approach has an advantage on both storage and retrieval.
For storage, the SingleStore columnar format leads to extreme compression of highly-regular JSON — like the responses returned by the Visual Crossing API. The compression means I can store whatever data I require and not worry about running out of space, even on the free shared tier. And everything about the schema is automatic - if I later decide my views require additional weather information like “snow depth” and “wind speed” I can just request them in my API call to Visual Crossing and the database will automatically adjust.
For retrieval, SingleStore only reads the requested fields of JSON/BSON from disk, even if the stored document contains many more. This means that my views that don’t contain “snow depth” and “wind speed” pay zero performance penalty for the presence of those fields in my database. This is a major difference from a more row-oriented storage mode and will lead to faster views in future versions of the app.
Accessing SingleStore
My app is based on NextJS, a JavaScript-based framework. I find the MongoDB NodeJS driver the most intuitive JavaScript client for working with JSON/BSON data. SingleStore supports a MongoDB-compatible API called SingleStore Kai that allows me to use the regular MongoDB NodeJS driver for all interactions with SingleStore.
SingleStore Shared Free Tier
SingleStore has a new Free Shared Tier that provides 1 GiB of data storage and no request metering. With the columnar JSON/BSON compression, I will be able to store a large amount of data within the limits.
Server-Side
I use Vercel for hosting jasonthorsness.com so it was an easy choice to stick with it for weather.bingo. What I didn’t realize is how Vercel is a lot more for non-static sites! I have a lot more to write about the server-side functions in a follow-up post.
Client-Side
I use NextJS for the client-side just like on this site, with graphics adapted from a set kindly released by Visual Crossing itself.
Domain
Almost every weather-related domain is taken. I checked all the TLDs and ended up with… weather.bingo.
The End
Try it for your own location at weather.bingo. And consider trying SingleStore’s Free Tier for your own projects!
Feedback on this app or article? Share it with me on Twitter.