I recently finished my time in the PDM program with Bob as my coach. This post focuses on how I made a back-end read-only database using FastAPI and a front-end using Streamlit along with the challenges that came with the whole project.
Project Idea and initial planning
This wasn’t my first attempt. I tried making this app with Django a year and a half ago, which I let languish in my private Github repo. My app idea was to make a game information wiki for Dragon Warrior Monsters, a Gameboy Color game that was released in Japan in 1998 as Dragon Quest Monsters – Terry’s Wonderland. It is a spin-off game based on the famous JRPG series, Dragon Quest.
You might be wondering, “Can’t you search for the info?” or “Why make it? It probably already exists!” I was frustrated with what was already out there and wanted to make my own. Bob and Julian mention in their Developer Mindset Training, Day 2 – App Idea ” ‘My app idea has already been done before!’ That doesn’t matter. It will be YOUR version + YOUR learning.”
For a beginner learning how to make app, why not make my own app for myself? Be selfish, do it for you.
Discussing my app idea with Bob, he recommended I try using FastAPI instead of Django since it would be a read-only database prepopulated with data with no planned updates. No user signups or logins, all great features in Django that wouldn’t be used anyways. We formulated a plan to have a separate back-end and front-end repo so any front-end framework could be used.
My main takeaway from that initial meeting was, “What’s FastAPI?”
FastAPI
FastAPI is described in their documentation website as “a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints.” Bob and Robin Beer had previously interviewed the creator of FastAPI, Sebastián Ramírez, on the Pybites Podcast #080 – Documentation-driven development with Sebastián Ramírez and there is a FastAPI learning path building a food tracker on Pybites.
For a newbie in web frameworks who only knew about Flask and Django, I had no idea FastAPI existed until Bob mentioned it. The tutorial documentation was superb and easy to follow. It covered how to create new endpoints, run it locally, and explained basic concepts well. However, the main challenge was creating the data models. This section isn’t really about FastAPI, it’s about how I struggled with SQLModel.
SQLModel
SQLModel is a library built on top of SQLAlchemy designed to interact with a SQL database using Python with the help of type hints and type annotation using Pydantic. It was also built by Sebastián Ramírez and designed to be used with FastAPI.
The database schema I had planned had some complicated relationships. Here are the main database relationships:
- One-to-many relationship
- monster_detail to family table
- Many-to-many relationship
- monster_detail to skill via skill_association table
- Many-to-many association table with multiple foreign keys from the same tables
- monster_detail and family to breeding table. This one is a little complicated to explain.
- Self-referential table — a table that contains a foreign key referencing itself
- skill.upgrade connecting to skill.id
The tricky portion of the data modeling was the last two bullet points. SQLModel did not explain how to handle multiple foreign keys in an association table from the same model nor did it explain how to create a self-referencing model.
In both cases, I needed to look into SQLAlchemy and use its more explicit relationship arguments. It took awhile to figure out the solution and I’m thankful that the problems had been documented and answered in a Github issues thread and a Stackoverflow thread. Great to know other users had the same problem.
Streamlit
After finishing the FastAPI back-end and marveling at code that worked after wrestling with SQLModel for a long time, I had to figure out a front-end. The initial plan was to use ReactJS because it’s a popular front-end framework in demand. After looking into some basic tutorials, I became convinced it would take much longer to push out a minimum viable product considering the time left I had with the PDM program and my limited knowledge of Javascript.
Bob sent a message about using Streamlit, which I was initially a bit doubtful of. Streamlit advertises it can “turn data scripts into shareable web apps in minutes. All in pure Python. No front-end experience required.” It’s great for data visualization and the apps featured in the gallery are amazing, but could it be used just to display word text in a table or a template? Absolutely.
I found a quick tutorial on how to build a basic website using Streamlit by Sven, a Python Youtuber who focuses on using Python in the workplace. He showed how Streamlit can be used to display basic info how you want. This convinced me to give Streamlit a shot.
Again, a well-written documentation website helped me find the Streamlit commands I was looking for. The Python Pandas library was also used since dataframes were much more convenient. The FastAPI endpoints returned JSON data, but it was easier to take bits and pieces of necessary data from an endpoint and create my own dataframe to display in Streamlit.
The main challenges of this project was creating a HTML hypertext link from a list page that links to a details page and a query filter that updates the displayed information. Both problems again were asked by previous users in discussion threads on the Streamlit community forum making my job a little easier.
Streamlit Multipage apps
Multipage apps is a relatively new feature introduced to Streamlit in June 2022. It lets the user create individual pages for an app that does several different things. Previously, the community used work-around methods to make multiple “pages.”
This feature was used to create different pages for the different endpoints from my FastAPI project. Each page is a different file within the project directory so the code is separated and organized. On the app, the page links are displayed in a sidebar menu.
Deployment
To use both projects, I had been running them locally in my web browser. The FastAPI URLs must be passed into Streamlit to use the FastAPI endpoints and both servers must be up and running. This is great if you’re making changes in both projects simultaneously, but it’s a hassle to always run both if you’re completely focused on the front-end. I decided to deploy my FastAPI on Deta Space.
Deta Space
FastAPI recommended using Deta Space, a cloud server that strives for easy and free deployment. I specifically used Deta Micros, a micro server that support read-only SQLite databases. Using Deta’s CLI, I deployed the working directory of my opened FastAPI project with relative ease. Updating is also done with the Deta Space CLI, so any changes made in the local folder must be redeployed.
Streamlit Cloud
Once satisfied with the look of the Streamlit app, deploying it was ridiculously easy with Streamlit Cloud. After signing up, all you do is point to the main file path from your Github repository. The great part about Streamlit Cloud is that it monitors the specific Github branch selected during deployment. When that branch is updated via git push, it updates the app. No further steps needed after pushing code.
The only downside with this service is the up time. After 7 consecutive days of no traffic, the app is automatically put into hibernation. A warning email is sent after 5 days of no traffic to your app. If in hibernation, anyone can ‘wake up’ the app, but it does take a couple of minutes to boot up.
The Dragon Quest Monster Database can be accessed at the following URL, https://dqmonstersdb.streamlit.app. If the site is in hibernation, just wake it back up to view.
Looking Back
I had a great time building both of these projects and there were places where it would’ve taken longer to accomplish without the help of Bob, my PDM coach.
Some lessons learned:
- Never use multiple foreign keys from the same table in SQLModel. This leads to a AmbiguousForeignKey error and force the user to shift through SQLAlchemy settings. Setting up multiple foreign keys is much easier in Django.
- Great documentation is king.
- Streamlit may not be the end product, but it’s a great tool to quickly create a working mock-up.
- Don’t doubt Bob.
I’ve shipped both of these projects out as a minimum viable product, but it doesn’t mean I’m finished with them. I have a list of features I would like to improve and add on, but that’s for another time. There are other project ideas I want to tackle and I will one day circle back to this project with more experience.
To the person just starting to learn and build projects, remember that it’s definitely about the journey of tinkering, failing, persisting, and then succeeding. Enjoy it.