03 Apr 2021
Between lines of code and a website lies the world of
DevOps: “Development Operations.”
Arguably, DevOps is the most overlooked—and challenging—parts of the software
development lifecycle. Nevertheless, it is still an equally crucial process, and
it involves everything from version control to build to deployment and scaling.
The DevOps cycle is not separate from the code: It is a part of the code.
For jadeocr, we used Git for our version control
system and GitHub as our remote repository and
project management hub. The platform is where we keep track of issues and manage
collaborating on code. We also use GitHub as part of our
CI/CD pipeline
to automatically build the website and server and make the changes live on the
website.
Pushing code to the master branch of our GitHub repository triggers numerous
GitHub workflows that comprise our CI/CD process. GitHub first sends a ping
across the web—an HTTP POST
request—to our server. After receiving the request from GitHub, a continuous deployment
server written in Node.js—previously in Golang—quickly goes to work, pulling
the latest code from the GitHub master branch and uses
Docker Compose to build and start the
application containers,
which are isolated environments called in which application code can run. The
server simply executes a series of shell commands after it receives a request
at a particular API endpoint; from a
technical perspective, therefore, this system is quite simple.
Afterward, a GitHub action builds the jadeocr-next client and the server into
their respective Docker images and publishes them to the
GitHub Container Registry.
These containers solve the common issue of “it works on my machine” and make
building, deploying, and scaling applications with
Kubernetes relatively straightforward
. Although I faced several (frustrating) hurdles while writing the
initial configuration files for these Docker containers, managing them after
the initial setup process has been a breeze.
Perhaps the most complex individual cog in this software
development machine is Nginx (Engine-X)
server and reverse proxy. Nginx is simply a web server, but it can also function
as a reverse proxy
to direct web traffic to its intended destination. We use Nginx in two places:
in front of the website’s static files and in front of the entire website.
When we build the website files into the HTML, CSS, and JavaScript that the
browser renders after loading jadeocr-next, our web
framework’s build tool, Vue CLI, creates a directory
of static files that must be served by a web server. Therefore, we use Nginx in
front of the client to serve these static files. We then use another Nginx
server in our cloud virtual machine to direct API requests to our web server
and website requests to the aforementioned client Nginx server.
Although each system is fairly simple on its own, the confluence of all these
tools in a single pipeline creates many moving parts which are challenging—but
fun—to manage. Perhaps such a DevOps system is overkill for such a small
application, but it provides jadeocr-next with the ability to easily scale and
avoid future errors. Most importantly, however, going through this DevOps
journey has been a valuable and fulfilling experience for me—even if I do
sometimes spend multiple days staring at a terminal window in confusion.
:wq
but we do plan to deploy a Kubernetes cluster in the future.
Tags: [
tech
software-engineering
]
14 Feb 2021
“Hmm, I should probably start a new project. Now, what on earth should I do?”
The description of this article was somewhat a lie: In reality, both execution
and ideation are extremely hard (or are they?). So many of us—including me—have
decided to start a project out of the blue, spending hours considering possible
problems to solve and whether or not an adequate solution already exists. The
project, naturally, ends in frustration and with nothing to show in the end.
When starting on a project, the most important element is an organic idea.
Regardless of how genius or foolish the idea is, I have found that the most
enjoyable, interesting, and impressive projects stem from organic ideas.
“Great, so what exactly is an organic idea?”
The best ideas are ones that come “out of the blue,” that is, ideas that are not
forced by a brainstorming session or hours of sitting in front of an empty
document and whiteboard. According to Barbara Oakley, an engineering professor at
Oakland University and a meta-learning educator, when the brain is not focused
intently on a particular task, it “enters into a natural default mode of diffuse
thinking” . In this state, the brain is free to generate new ideas by
connecting previously acquired knowledge and observations.
Most of us are already familiar with this state: we know that studying a unit’s
material the night before an exam is not conducive to scoring well, and we also
know that being repeatedly exposed to information—say, words in a foreign
language—is an excellent way to learn. In fact, jadeocr
leverages a similar effect.
Jadeocr illustrates the importance of organic thinking in two ways: The app
encourages the mind to naturally develop a vocabulary base through the
aforementioned spaced-repetition,
technique, allowing users to slowly acquire foreign language vocabulary
while giving enough time between review sessions to allow the brain to engage
its diffuse mode. Additionally, jadeocr itself was a product of organic thinking.
The idea for jadeocr came to me during my fourth-period Mandarin class in my
sophomore year—notably, it was not the result of any brainstorming session.
I was simply sitting at my desk, practicing writing a handful of simplified
characters to prepare for a coming vocabulary quiz. I wasn’t worried about scoring
well on that particular quiz, but I wanted to earn a perfect score on
the upcoming midterm exam, and the only thing holding me back was a few writing
mistakes.
As an aside, writing Chinese characters requires some attention to
detail; some notorious examples are 末/未, 复/夏, 换/挽, 口/囗. The last one,
to be fair, has a radical,
but my point still stands.
As I started to search for apps that could help me prepare for the next midterm
exam, I realized that each of them had some shortcomings that I was not willing
to deal with. Eventually, I came to terms with the fact that no existing app
satisfied my needs, so I set out on building my own with some help from a friend.
The idea was organic, and as a result I was genuinely passionate about the project:
I didn’t mind working on it for eight or more hours per day during my summer
break, and the result of my work was a product that I was proud of. Moreover,
I was willing to build it again—nearly
from scratch—to improve my code quality, add more features, and share it with
more people.
We all generate ideas this way: We all have those points where we think, “if
this existed, my life would be so much easier.” The two most difficult parts of
bringing these ideas to fruition are capturing the idea and getting started.
Capturing ideas—taking note of them when they occur—consists of two primary
steps: Recognizing the idea and capturing it. Recognizing ideas takes practice;
the next time an idea comes to mind—the next time you wish “I wish this existed”
or the next time you think “it would be cool if this was a thing”—make a mental
note of that idea. Thoughts are ephemeral, however, so an even better idea is
to make either a physical or digital note.
With ideas in hand, the hardest step is now getting started. Everything else
will fall into place after a project has started: Specifications, technologies,
and design can all more or less figure themselves out without too much effort.
This doesn’t mean building a project doesn’t require significant amounts of
time and effort—it does. However, with an organic idea, the passion required
to drive a project forward is often self-sustaining.
Tags: [
tech
thinking
]
