Back to projects

Jobbot

A real-time job monitoring system that scrapes postings and auto-accepts opportunities before they're gone.

Python Selenium BeautifulSoup Pushover API DigitalOcean
View on GitHub

The Story

My sister worked as a substitute teacher. Jobs post at random times (early morning, late at night, etc.) and get claimed in minutes. Miss the window, miss the day's pay. She was setting alarms, refreshing the portal between meals, staying up late just in case. I built her a bot that runs 24/7 and books jobs before she even sees the alert. Now she can sleep through a 6am posting and wake up to confirmed work.

BEFORE
  • Constantly checking her phone
  • Missing jobs when she wasn't looking
  • Waking up early just in case
  • Staying up late refreshing
AFTER
  • Wakes up to jobs already booked
  • Sleeps through the night
  • Stopped touching the portal
  • Works as much (or as little) as she wants

How It Works

01

Prepare

Selenium spins up Chrome and logs in. Before every job check, the bot makes sure the session is still healthy and self-heals if not. No silent failures downstream.

02

Check

Polls the portal on randomized intervals weighted around the hours jobs actually post. Random timing keeps the pattern from looking mechanical.

03

Match

BeautifulSoup parses each posting. The bot checks her filters (dates, classifications, blackouts) and takes the first job that works, since sub jobs go to whoever clicks first.

04

Act

Accepts the job and verifies the accept actually landed. A screenshot follows with the result. She wakes up to an answer, not a maybe.

System Architecture

One orchestrator, four single-purpose modules, two external services. The dashed border separates code I wrote from systems I integrated.

Tap or hover any module for the deeper trade-offs.

Job portal Selenium Job Bot Session orchestrator Main entry point driver_manager Chrome lifecycle error_handling Exceptions, retry timing Randomized waits notifications Pushover sender Pushover API job alerts errors + status User Admin (Jen)

How It Stays Running

She stopped checking the portal manually. A silent failure wasn't just a bug, it was her missing income.

I was self-funding this on a $6/month DigitalOcean droplet (1GB RAM), and the bot couldn't leave a mess. Stale browser sessions, leaked Chrome processes, orphaned profile data: any of those would eat the droplet alive over a long weekend.

So every recovery action picks the cheapest fix that fits the current state. Refresh first. Build a fresh driver only when the old one died (or on the 1st cycle of a new session). Destroy Chrome and wipe its temp directory every 10 cycles, before memory can creep. The only branch that doesn't loop is the one that needs a human.

prepare_session picks the cheapest fix for the current state Refresh page cycles 2–10 (default) Create driver 1st cycle, or if driver died Login 1st cycle, or if logged out Do the work scrape · match · accept (see "How It Works") Success Wait Runs 1–9 Run 10 Destroy driver contain memory leaks TempError 1st or 2nd attempt? 1st 2nd raise PermanentError PermError Page structure changed, credentials invalid, or retries exhausted needs a human Stop, notify both she accepts jobs herself while I fix it next cycle new session retry session

Either the bot accepts the job, or she does. Either way, she doesn't miss it.

How It Works For Her

Designed around her, not the code.

What if she's sleeping and a job gets auto-accepted she doesn't want?

Added a same-day toggle. Off by default. If she wants a job today, she turns it on. If it's night and she hasn't gotten one and wants to try early tomorrow, she keeps it off.

What if auto-accept fails but the job is still there?

Notify her first when the job appears, then try to accept. If it fails, she still knows the job exists and can grab it herself.

What if it's a job she doesn't want, but she'd take it today?

Still notify her about all jobs, even ones outside her preferences. Maybe she doesn't have anything else lined up.

When the bot breaks, who gets the alert?

Two notification tiers. I get the engineering errors (timeouts, missing buttons, retry storms). She gets only what she can act on (a job appeared, accept failed, the job's gone). Her phone stays a job-finder, not a debug log.

What I had to solve

I couldn't see what the bot was doing in real time.

It ran headless on a server I wasn't watching, parsing a page that only existed at the moment a job was posted. So throughout development, the bot saved the HTML it was pulling and took screenshots on every check, and I'd review them to confirm what the bot saw matched what was actually there. I added an in-page check for the "no jobs available" text so an empty result was confirmed, not assumed. My sister kept checking the portal manually as ground truth until I was sure the bot's signal was reliable enough to take her out of the loop.

The platform was blocking my bot.

Jobs weren't showing up and at first I assumed I had a bug. I didn't. Out of the box, headless Chrome announces itself: the navigator.webdriver flag, the enable-automation switch, and request timing too regular to be human. After some research I tested the bot-detection theory: stripped all three indicators in Chrome options, plus randomized the wait times in the timing module so the polling pattern didn't look mechanical. Jobs started flowing in.

I kept running out of memory on the 1GB droplet.

A 24/7 program with no headroom, so I had to bound every leak instead of buying a bigger box. Every part of the system had to fit a memory budget: I swapped Chromium for Chrome (more stable under load), destroyed and rebuilt Chrome every 10 cycles so its memory couldn't creep, wiped its temp profile each time, and deleted debug HTML and screenshots the moment I'd sent them. A bigger droplet would have delayed the same leak, not solved it.

It was my first time managing my own server.

Someone started hammering SSH on port 22 (the default) until the box ran out of memory and went down. I learned quickly that the defaults aren't safe. Moved SSH off port 22 (they always try that one first), set up an SSH key so I could log in without a password, and disabled password authentication entirely. The attacks didn't stop, they just stopped mattering.

The Result

My sister doesn't check the job portal anymore. She gets a notification, and the job is already hers. The system has been running reliably, and she hasn't missed a job since.

What I Learned

I learned to run what I build, not just write it.

Provisioning the server, securing it, and keeping a long-running process alive on 1GB of RAM is its own skill. When a brute-force attack took it down, hardening the box (off the default SSH port, key-only login) was on me.

I learned to verify, not assume.

Clicking "accept" isn't the same as getting the job, so the bot checks for the confirmation before it tells her she's booked. A bot that assumes it worked is worse than no bot.

I learned to treat failures differently depending on what they are.

A network blip should retry; a changed login page should stop and tell me. Sorting errors into recoverable versus terminal, instead of one catch-all, is what made it trustworthy.

I learned that most of the work is turning a person's needs into rules.

Her real what-ifs (only same-day jobs, a heads-up before it accepts) had to become actual logic. Requirements live in someone's head; getting them into code correctly is the job.