In the late 1990s, I eagerly bought the book
“Web Client Programming with Perl”
and used the LWP
library to scrape the Web in automated fashion. I continued doing that
into the 2000s. I am happy that nowadays, I can just use Haskell to do
this kind of programming, in a succinct way also.
Today’s topic is wreq,
Bryan O’Sullivan’s high-level
library for doing Web client programming designed specifically for
usability.
wreq makes use of the
aeson ecosystem for JSON
and lens and ecosystem,
including
lens-aeson, so you
may want to check out Ollie’s 2012 Days of Hackage posts on
aeson
and lens.
Since wreq already has an extensive
tutorial and reference documentation,
I’m not going to repeat its explanations. Instead, I’m going to give an
example of use that should be simple enough to be understood from
context, then discuss the issue of using operator syntax in Haskell.
I spent my formative years writing software before “testing framework”
was in my vocabulary, before
“test-driven development”
was a thing. I shudder to think of those years, because now I’m a
believer in tests and even in test-driven development (TDD), according to my
interpretation of what that means (since everyone has a different
definition).
There are a bunch of testing tools that have been available in the
Haskell ecosystem for some time. In fact, Ollie in his “24 Days of
Hackage” covered
Don’t laugh, but once upon a time, I made Perl my main programming
language of choice (between around 1999 and 2010). There were many
reasons for this, but one reason was that Perl made it very easy to do
text processing using regexes.
If you are a seasoned Haskeller, you might be thinking, “Why not use a
real parser instead?“, such as the venerable
parsec, which was covered in a
2012 day of Hackage?
(Or, today, one could consider one of several other newer alternative libraries
for parsing. A later day of Hackage will say more about this!)
After all, Jamie Zawinski famously once wrote, “Some people, when
confronted with a problem, think ‘I know, I’ll use regular
expressions.’ Now they have two problems.” I even gave a talk at
Pittsburgh Tech Fest in 2013,
“Stop overusing regular expressions!”,
in which I promoted writing parsers rather than writing regexes.
But, sometimes I do want to use a regex. In that case, I have been
using an obscure but useful package, pcre-heavy.
Today I’ll show how to use pcre-heavy, and while at it, also show
how to ship one-file standalone Haskell scripts that only require
Stack.
A couple of days ago, I happened to see a
tweet from Ollie Charles
that he didn’t have time to do his usual annual December “24 days of…”
Haskell blog posts this year (2015) and felt sad because I’ve
learned a huge amount from
reading them. In both 2012 and 2013, he
wrote “24 days of Hackage”, daily short and sweet blog posts that
showed how to use selected Haskell packages you can get from the
community archive Hackage, and in 2014
he covered GHC language extensions.
With some trepidation, I decided that I would do a “24 days of
Hackage” series myself to cap off this year, to share a selection of
the huge number of Haskell packages I find useful. I thought it would
be particularly appropriate to do this given that 2015 was the year
that I migrated to using Haskell as my main language for most new work and
personal projects, and therefore this has been a year of considerable
discovery for me.
This presentation is part of an experiment, a big shift in how I’m
trying to talk about not only “functional programming”, but any kind
of programming and more generally any topic at all (whether it’s
cooking or music) for a wide audience. As I’ve been doing more
teaching and collaboration in many areas of my life, I’ve been trying
to figure out ways to better reach people. In particular, when
presenting something new, it’s useful to motivate it and somehow
connect it to something they already know, even if it’s different.
I’ll write more later about what I did differently in this talk than
I’ve done in the past, and why. For now, I’ll just say that one big
addition I made was that I prepared feedback forms to be filled out,
both to help me understand what I can do better and also to help
Pittsburgh Functional Programming plan further events and topics that
people want to see. So I will summarize the feedback I got (fifteen
participants filled out feedback forms).
By sheer accident, while I was looking for something else online, I
came across a Web site for Aegis, a
distributed version control system I used at work back in 1995-1997. I
was surprised this software was still alive after twenty years. A lot
of software has a very short life span, and since I had not heard
anything about Aegis all this time, I would have guessed that it had
died.
I remembered that we had used this software along with another tool
the author developed, called
Cook, which was a
replacement for
Make. I was
unsuccessful in locating any official Web site for Cook, so I assume
it is more or less dead.
Meanwhile, I found by coincidence that actually, the author,
Peter Miller,
died less than a year ago (July 2014). There was enough memory of him
that someone wrote that Wikipedia page on his contributions to open
source software. However, his Web site http://www.canb.auug.org.au/~millerp/
, sadly, has died with him, and with that, the site he had on Cook.
But we know that the Web is “forever”, right? Let’s see.
Until now, I haven’t been publishing anything on any of my three blogs
for half a year now. There are many reasons, but one of them was that
I wanted to migrate away from
Octopress 2. Octopress 2 is ancient and slow
and unmaintained, and I’d been waiting for
Octopress 3 for over three
years
to arrive,
so when I heard that Octopress 3 was finally going to be officially
announced at JekyllConf, I decided it was
time to migrate my blogs, to Octopress 3 or
Jekyll, or something else entirely.
The Rust programming language, which is nearing its important version 1.0 release, but already seen a lot of use, has many interesting features, the most prominent of which is its ownership static type system that prevents certain kinds of bugs common in many programs in C or C++.
For example, it is common in most applications to create graphs of objects pointing to each other, in which it is not clear how many pointers point to a particular object, who is to free the object’s memory when it is no longer needed, and what happens to all outstanding pointers to that block. Languages supporting precise garbage collection use various algorithms to determine when memory can be freed, and have become particularly popular in the past twenty years, but there are still situations in which garbage collection is not ideal, hence the continued relevance of languages such as C++.
When I was working as a software engineer in the 1990s developing desktop applications with user interfaces for the X Window System, we used frameworks that included C++ smart pointers that used reference counting to handle graphs of interconnected data. The C++ standard itself lagged behind in standardizing smart pointers; it started with the terribly flawed and unusableauto_ptr that finally got deprecated in C++11, then moved on finally to unique_ptr and shared_ptr and weak_ptr.
When talking with C++ programmers about Rust, I have found that often they have been puzzled about how Rust’s ownership system does anything better than what C++ unique_ptr already does. Here is an explanation (I will not be discussing analogues of shared_ptr and weak_ptr here).
