Tag: servers

Posted on
by

My brother insists that I write a blog post about this, even though it’s really just basic Unix stuff and not anything the least bit esoteric, or even related to Syncthing really.

I didn’t want to sync my entire music library to my phone, because it’s huge, and about half of it is just stuff that was either of merely passing interest, or really no interest at all. So, I decided to create a folder I called “Core music,” that would contain the other half of my music library—all the stuff I might actually want to listen to. However, I didn’t want to make copies of 7 GB of music files, because that would be crazy.

My first thought was just to create a folder with symbolic links to the music I wanted to sync to my server (and thence to my phone). But that didn’t work at all. (That is, Syncthing just syncs the symbolic links, and since the music wasn’t on my server, on the server they didn’t point to anything.)

My second thought was to make hard links, but even though I’m a Unix nerd from way back, I initially started by trying to make hard links to the folders that had the music I cared about, because it has been years since I did this sort of thing. Of course that didn’t work at all. (You can’t hard link to a directory, for reasons.)

Finally, I created this little function:

function linkmusic { pax -rwl '$1' ~/Music/Core\ music; }

Then I could invoke it by cding into my iTunes Music folder (where the directories actually are) and then invoking the function for each folder I wanted in my “Core music” folder. (My natural inclination is to go to the folder where I want the music to end up, but then the pax command didn’t do what I wanted, because it wanted to copy the whole directory structure, but I just wanted to copy the directory structure starting at iTunes Music, so that’s where I wanted to be when I invoked my function.)

So, let’s hope that Steven is happy now that I’ve documented this.

Posted on
by

A year and a half ago, my brother gave me a Raspberry Pi 3 as a birthday present, suggesting that I should use it to run my own server.

I used to run my own server. A friend who liked to build such things had built it. It had two ethernet ports, one connected to my cable modem and the other connected to my WiFi router, and it was running OpenBSD (then the most secure OS easily available) and was configured to serve as a firewall.

I used it as a server in other ways. I put an extra disk drive (40 GB!) in it where I could store files that I might want to access from elsewhere. (In particular, when I went to Clarion I copied my latest draft of my current story there each evening, in case of catastrophic computer failure.)

It didn’t require much upkeep, but it required more than none—which turned out to be more than I wanted to devote to it. At some point a serious security flaw was discovered in the OpenBSD release I was running. By then most desktop machines had built-in firewalls as did most routers, and I had Time Machine as a backup solution. It seemed safe to give up my server, and easier than updating it.

In the years since then, the use of cloud services has become ubiquitous, to the point that practically everything I do ends up in the cloud—my photos go to both Flickr and Google. I also use Dropbox (where I have Scrivener stash a backup copy of everything I’m writing) and I stash some amount of my music at both Google and at Amazon.

That’s all great—those services are well backed-up, and the servers are very likely running the latest security patches—but I really like the idea of having my own data on my own machines. But I want that without giving up the advantages of having my data in the cloud. Hence wanting to have my own server.

All that as prequel to my brother coming to visit this past week, and helping me get my Raspberry Pi server up and running.

Once the basic install of Raspbian was up and running, I went ahead and ordered a bit of hardware for it. I got a short ethernet cable to connect it to my router, so that it doesn’t have to do WiFi for basic connectivity (although WiFi and Bluetooth are built in). I also got a slightly more powerful USB power supply for it, mainly because I also got a portable USB hard drive that takes its power from the USB port, meaning that the power needs to be available to the Raspberry Pi. Finally, I got a case for it, so that I don’t just have a naked circuit board sitting on my dresser.

This time the hard drive is 1 TB rather than 40 GB.

For cloud functionality I’m following my brother’s example and running syncthing, which has the advantage of being able to handle being behind a NAT and not having a port exposed to the outside world. I’m running it on my Android phone as well and sharing my photos with a third place: my server. The server then shares them with my desktop machine, so they’re available to use. (That’s how I got the photo above: Taken with the phone and then transferred to the desktop within about a minute.)

I’m still sorting out my sharing strategy. I don’t want to share my whole Music folder with my phone, because it would use all the space there. (I’ll probably end up making a folder with an “essential subset” of my music to share with the phone.) I don’t think I want to share my whole Documents folder on my desktop machine, but I’m not sure yet. For the time being I’m sharing a folder I call “Active writing” with the files I’m currently working on, on the desktop, the server, and my laptop. That way they’ll be available wherever I want to work on them.

Other things are tougher. I’d like to have my own calendar server, but that doesn’t seem easy. I should go back to my post on the google-free option and see what else I was thinking about that I might now be able to implement.

For now, though, I’m pretty happy.

My previous server was rack mount width and maybe four or five inches tall, about the size of a stereo component. This one is maybe 3 inches by 5 inches, rather smaller than the hard drive it’s sitting on.

Posted on
by
Student-built supercomputer at the National Petascale Computing Facility
Student-built supercomputer at the National Petascale Computing Facility

I ran my own server for a while. It was an OpenBSD box running on a cheap 386 board in a re-purposed PC case with an extra ethernet card. It sat between my cable modem and my home network and acted as a firewall. It also provided a few services to me in the outside world. In particular, it ran a little program for tunneling ssh traffic through the http hole in the corporate firewall, so I could get into my home network from work. (It was not, let me be clear, the supercomputer shown in the image to the right.)

I turned it off several years ago. Desktop computers got more secure, so the firewall was no longer necessary. I quit working at a regular job, so I didn’t need to tunnel my ssh traffic any more. But the main thing was that external firms started providing the sort of services that had previously made it seem worth going to the trouble of running your own server.

I use a bunch of those services. I share photos at Flickr. I host this website at Dreamhost. I post things on Facebook, Google+, and Twitter. I read RSS feeds using The Old Reader (and share things there as well).

I’d previously thought that it would be best to have my own server for all these things—in particular sharing stuff I wanted to share—my writing, my pictures, my calendar items, etc. But the commercial services were better than what I’d have had if I ran my own server. Flickr provides a much better gallery than I’d have managed to put up, if I’d had to host my own. (The idea of serving—and owning—your own data was the impulse behind Diaspora as well, of course.)

Just lately, though—especially since Google announced that they were shutting down Google Reader—I’ve begun to rethink things.

If I ran my own server, I wouldn’t have to worry that some giant company would abruptly decide that providing some service I was using “no longer aligned with corporate priorities.”

I’m not in any hurry to move from this new thinking to actually running my own server again. For one thing, it wouldn’t make any sense to try to run a public-facing server at home over a consumer-grade home network link. (Although maybe one of the higher-grade packages through UC2B would be good enough.) But I am thinking about it. I don’t like any of the calendar services out there; maybe running my own calendar service, just for me and my family, would be just the thing.

In any case, running a server would be a lot easier now than it was back when I did it before. The hardware is cheaper and faster. The software is more reliable and easier to use. Before, I had to painstakingly build everything. Now I could just do a quick install on a Raspberry Pi, maybe with Freedom Box software.

I’ve always known that with “free” corporate services I’m not a client; I’m a commodity being pimped out to advertisers and others. I’ve tolerated it, because the “free” services are often pretty good—better than I could manage if I had to roll my own. But it’s always bugged me. Now, between the hardware and software for rolling my own getting cheaper and better, and the increased visibility of the consequences of going with free services that get can get turned off on corporate whim, maybe I’ll get it together to make the jump to my own server once again.