Architecting Flow in Elixir Programs: An Introduction
Getting the “flow” of a program is one of the first things I do when looking at open source software or joining a new project at work.
The easier it is to grasp how data flows through a program, the easier it is for me as a developer to estimate the impact of changes (and the business of software is all about changes; while “writing hot new cool stuff” is most certainly fun, most professional work I’ve done falls into the “maintaining once-hot-new-cool-stuff and adapt it to changed requirements” category).
Now, writing CLIs (command line interfaces) is a personal pet peeve of mine. These are little, mostly straight-forward programs, sometimes fulfilling a single purpose (like cat or touch) and sometimes being the fascade for a host of features (like git, docker or mix).
In any case, CLIs are a perfect example to demonstrate what I would consider good “flow” (and, naturally, the ideas presented in this post are just as applicable to embedded software, web applications or any other program).
Let’s see some code
Imagine a CLI for converting images. The flow might look something like this:
Implemented as a Mix task, it could look something like this:
defmodule Mix.Tasks.ConvertImages do
use Mix.Task
def run(argv) do
{opts, args, _invalid} =
OptionParser.parse(argv, switches: [target_dir: :string, format: :string])
glob = List.first(args) || "./image_uploads/*"
filenames = Path.wildcard(glob)
target_dir = opts[:target_dir] || "./tmp"
File.mkdir_p!(target_dir)
format = opts[:format] || "jpg"
# TODO: ensure valid format
results =
Enum.map(filenames, fn filename ->
Converter.convert_image(filename, target_dir, format)
end)
IO.puts("Wrote #{Enum.count(results)} images to #{target_dir}.")
end
end
# NOTE: we will omit the definition of `Converter.convert_image/3` for now and
# assume it works as one would expect (take a filename, convert its
# contents to the given format and write the result to the given target
# directory).
I have written this kind of program a dozen times before. There’s nothing really wrong with it, except that writing these kinds of throw-away scripts is much more fun than inheriting them. So let’s do our successor (or our future-self) a favor …
We could group related work inside the function if we want to illustrate the flow of our program:
defmodule Mix.Tasks.ConvertImages2 do
use Mix.Task
def run(argv) do
# 1 - parse options
{opts, args, _invalid} =
OptionParser.parse(argv, switches: [target_dir: :string, format: :string])
glob = List.first(args) || "./image_uploads/*"
target_dir = opts[:target_dir] || "./tmp"
format = opts[:format] || "jpg"
# 2 - validate options
filenames = Path.wildcard(glob)
if Enum.empty?(filenames) do
raise "No images found."
end
unless Enum.member?(~w[jpg png], format) do
raise "Unrecognized format: #{format}"
end
# 3 - prepare conversion
File.mkdir_p!(target_dir)
# 4 - convert images and write them to target directory
results =
Enum.map(filenames, fn filename ->
Converter.convert_image(filename, target_dir, format)
end)
# 5 - report results to STDOUT
IO.puts("Wrote #{Enum.count(results)} images to #{target_dir}.")
end
end
But this might just be a case of “commenting not-so-ideal code”, so let’s put these sections into separate functions:
defmodule Mix.Tasks.ConvertImages3 do
use Mix.Task
@default_glob "./image_uploads/*"
@default_target_dir "./tmp"
@default_format "jpg"
def run(argv) do
{glob, target_dir, format} = parse_options(argv)
validate_options(glob, format)
filenames = prepare_conversion(glob, target_dir)
results = convert_images(filenames, target_dir, format)
report_results(results, target_dir)
end
defp parse_options(argv) do
{opts, args, _invalid} =
OptionParser.parse(argv, switches: [target_dir: :string, format: :string])
glob = List.first(args) || @default_glob
target_dir = opts[:target_dir] || @default_target_dir
format = opts[:format] || @default_format
{glob, target_dir, format}
end
defp validate_options(glob, format) do
filenames = Path.wildcard(glob)
if Enum.empty?(filenames) do
raise "No images found."
end
unless Enum.member?(~w[jpg png], format) do
raise "Unrecognized format: #{format}"
end
end
defp prepare_conversion(glob, target_dir) do
File.mkdir_p!(target_dir)
Path.wildcard(glob)
end
defp convert_images(filenames, target_dir, format) do
Enum.map(filenames, fn filename ->
Converter.convert_image(filename, target_dir, format)
end)
end
defp report_results(results, target_dir) do
IO.puts("Wrote #{Enum.count(results)} images to #{target_dir}.")
end
end
It’s getting easier to see what is happening and what phases the program walks through to reach its goal.
If we revisit our diagram from the top, we start to see that we added the activity of validating our inputs:
See below how we can adapt this diagram into code using Elixir’s pipe operator (|>).
defmodule Mix.Tasks.ConvertImages4 do
use Mix.Task
@default_glob "./image_uploads/*"
@default_target_dir "./tmp"
@default_format "jpg"
# NOTE: we could also have refactored this using `with`, but
# it doesn't really matter for the point I'm trying to make ^_^
def run(argv) do
argv
|> parse_options()
|> validate_options()
|> prepare_conversion()
|> convert_images()
|> report_results()
end
defp parse_options(argv) do
{opts, args, _invalid} =
OptionParser.parse(argv, switches: [target_dir: :string, format: :string])
glob = List.first(args) || @default_glob
target_dir = opts[:target_dir] || @default_target_dir
format = opts[:format] || @default_format
{glob, target_dir, format}
end
defp validate_options({glob, target_dir, format}) do
filenames = Path.wildcard(glob)
if Enum.empty?(filenames) do
raise "No images found."
end
unless Enum.member?(~w[jpg png], format) do
raise "Unrecognized format: #{format}"
end
{glob, target_dir, format}
end
defp prepare_conversion({glob, target_dir, format}) do
File.mkdir_p!(target_dir)
filenames = Path.wildcard(glob)
{filenames, target_dir, format}
end
defp convert_images({filenames, target_dir, format}) do
results =
Enum.map(filenames, fn filename ->
Converter.convert_image(filename, target_dir, format)
end)
{results, target_dir}
end
defp report_results({results, target_dir}) do
IO.puts("Wrote #{Enum.count(results)} images to #{target_dir}.")
end
end
Use Your Imagination
The example is meant to be easily accessible and relatable.
Please imagine the presented solution for a complex app or just a non-trivial use-case for the program above, like converting the images to match certain dimensions based on their filenames, reporting errors, adding a verbose flag to give more information to the user during runtime, optionally writing the EXIF information of the original images to an external datastore and/or serving the collected image metadata through the same CLI while lazily converting new images any time the program is run.
Now we’re talking.
Requirements change, software needs to be maintained
One thing that immediately stands out in the examples above: the last version is the most verbose one (the first version was 24 lines, the last one clocks in at 62 lines).
If we assume a more complex example, this difference in lines will become less significant. In these cases, we will reap the benefits of having a more approachable codebase, cleaner stacktraces and an easier time to delete old and add new code.
The last point is paramount because requirements for software change all the time. So we want to make our programs as adaptable to change as possible.
A clear flow can enable just that. 👍
