Josh's Toolbox

netcat: Making a very simple web server

I’ve recently toyed around with using netcat to make sharing data between machines easy. In this post, I cover how netcat works and how I’ve been using it.

If you want to share data across computers, it can be surprisingly cumbersome. If both system are in the same ecosystem, like Apple’s, Google’s, or Dropbox’s, it can be more straightforward. If you’re on the same network, you can use python’s http.server module to share a whole folder of files.

> python -m http.server

And there are tools like copy-party (which I’ve been meaning to mess with). And of course pastebin. But all of these require some amount of setup and/or turning over your data to a third party. Python’s may be the simplest, but it does require turning your data into a file (and installing python).

But there is a tool that’s designed to just serve and receive arbitrary data across UDP or TCP connections and may already be on your system: netcat. Let’s take a look at a simple transfer of information across based off the example from man 1 nc.

> echo "Hello world!" | nc -l 8080 &
> nc -N -d localhost 8080
Hello world!
[1]+  Done                    echo "Hello world!" | nc -l 8080

That trailing line is bash announcing that the background process has completed, successfully in this case given by the Done string. If it had failed, it would have instead said Exit n where n is the exit code the process finished with. I won’t be showing it in future code snippets.

Let’s breakdown what just happened.

Our first invocation sets up the listening server. We pass the string Hello world! to it via a pip (|) to netcat (nc) which has been configured to listen (-l) on port 8080. The & tells bash to run it as a background job so we can do other things with the current shell.

The second invocation makes a request for our TCP message. -N tells netcat to shutdown once it sees the end of a file and the -d tells it that there will be nothing piped to it, so that implicitly happens as soon as it receives data from the server. The localhost and 8080 tell it how to reach the server we just setup.

And boom, the message is written to STDOUT of the listening process.

But, this still requires netcat on both ends. When I was setting this up, I wanted to get arbitrary data to any of my devices on my local networks, and one of those is my phone. I don’t have a terminal on my phone and my fiancee runs iOS, so even if I did, if want to share things with her, this solutions wouldn’t work.

But, what all of our devices have is a way to understand HTTP message, our browsers. And HTTP is just specifically formatted text sent over TCP. On Linux, we also have curl, which will show HTTP responses directly in our terminal. So, let’s switch our receiver to the proper invocation:

> echo "Hello world!" | nc -l 8080 &
> curl localhost:8080
GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/8.5.0
Accept: */*

curl: (1) Received HTTP/0.9 when not allowed

Like I mention above, HTTP requires a specific format, and netcat has actually shown us a bit of what that looks like, at least on the requesting end. It indicates the request type GET, that path being requested /, and the HTTP version the request is using HTTP/1.1. The what follows are some header fields that can tell the server more about the request to help it handle things. An intelligent server will use that information to respond appropriately, but we can target the largely lowest common denominator and ignore all of that.

So, let’s serve a properly formatted HTTP message with netcat:

> printf 'HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 13\r\n\r\nHello world!\n' | nc -l 8080 &
> curl localhost:8080
GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/8.5.0
Accept: */*

Hello world!

Huzzah! We got our message.

Now, this looks a little weird. We’re using printf because HTTP expects lines in the frontmatter to be separated by a carriage return (\r) followed by a new line (\n). Here are the lines we’ve hard-coded:

Let’s resolve some of the issues here. First, so far we’ve been hard-coding our string and the length. So let’s handle that by putting this all in it’s own bash script:

#!/usr/bin/env bash

content="$(cat)"
contentLength="$(printf "%s" "$content" | wc -c)"

printf "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %d\r\n\r\n%s" \
  "${contentLength}" \
  "${content}" \
  | nc -l 8080

If we call this simple-share.sh and mark it executable ( chmod u+x ./simple-share.sh ), we can call it like so:

> echo "Hello world!" | ./simple-share.sh >/dev/null &
> curl localhost:8080
Hello world!

The >/dev/null means we won’t be seeing the request information in our output anymore.

Huzzah? There’s actually a caveat here, the trailing new-line is gone. Bash has actually dropped those when we assigned data to variables. Worse, if we ever included a null-byte (\0) as would 100% be possible and expected with arbitrary data, it’ll drop those too. So, we need to get a bit smarter and use an intermediate file:

#!/usr/bin/env bash

contentPath=$(mktemp)

cleanup() {
  rm --force "$contentPath"
}
trap cleanup EXIT

cat > "$contentPath"
contentLength="$(wc -c <"$contentPath")"

{
  printf "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %d\r\n\r\n" \
    "${contentLength}" \
    "${content}"
  cat "$contentPath"
} | nc -l 8080

Whew. Almost makes you rethink how complicated the other solutions are.

But, run everything again, and we’ve gotten our new-line! And arbitrary files won’t get messed up. Now, to get this to another machine, there’s a few more things you’ll need to do:

Then, you can use that combined information to go to it in the browser on your target device by filling in the address bar appropriately: http://IP_ADDRESS:8080. This all assumes you’re on the local network and transferring data across the Internet is WELL outside the scope of this blog post.

There are still some other deficiencies in this solution:

Those deficiencies aside, this solution can pretty easily be mapped to other applications. Just sprinkle a little bit of extra HTTP knowledge, and you can do things like make requests automatically go to the site of your choosing:

> printf "HTTP/1.1 307 Temporary Redirect\r\nLocation: %s\r\n\r\n" "https://joshstoolbox.com" \
    | nc -N -l 8080 &
> curl -L localhost:8080

Which brings us to an important point. There’s a -N in our netcat server call now. This is because we didn’t set the Content-Length header. As a result, the receiving HTTP client doesn’t know that the message is complete and it doesn’t know it should terminate the TCP connection. Only the server knows that there is no more data to send and must be the one to terminate the TCP session. If you added Content-Length: 0 to the headers, you wouldn’t need the -N flag.

In fact, in our very first example, if we sent too much data, it would end up truncated because the listener wouldn’t actually know when the data would stop being received and would close the connection prematurely. If we switched which call has the -N, everything would be transferred. And though HTTP clients don’t have this problem when headers are properly set, it’s still probably worth it to set it in our simple-share.sh script as well.

Now you have the power to make really simple servers with netcat and share arbitrary data from your Linux machine to other devices on your local network. One final note: after a single request, these server can’t share the information again. netcat sends data from STDIN and it has already consumed it all.

I hope you’ve learned something, and I hope to share a more refined version of this script and its siblings soon!