Our data analyst found rows in production where the outlet on a payments row is different from the outlet on the job that payment belongs to (BI question 349). The most extreme pair looked alarming: one payment tied to IKEA Alexandra whose job was at McDonald's Rivervale Mall, and a second payment with the exact mirror image.

We investigated all of it on 7 July 2026 — code, production audit trail, and the finance side with Shaun, Rhesa, Renz and Mei Chun. This post is the write-up so the whole team understands what happened, why the SOA was never wrong, and what we changed.

I have been reading about "AI agents" and "workflows", and about SDKs like flueframework, LangGraph, the Vercel AI SDK, and Ruby options like RubyLLM. I want to write down what these things actually are, in plain terms, so we can study it together and decide where it fits in Jod.

This post uses our own gig domain — Gig::Shift, Gig::PayRate, suspensions, auto-selection — as the examples. If you understand how we post a shift and pay a worker, you already understand most of what you need.

The one idea everything hangs on​

Take away the hype. A large language model (LLM) is a function that takes text and returns text.

output_text = model(input_text)

That is all it is. Three properties matter:

• It is stateless. It remembers nothing between two calls. Nothing.
• It has no side effects. It cannot read our database. It cannot send an email. It cannot charge a card. It only produces text.
• It is not deterministic. The same input can give a slightly different output.

Hold this picture. Almost everything people call "memory", "tools", "agents", or "RAG" is scaffolding we build around this one stateless text function to make it useful in a real business.

Two things follow directly:

• Because the model has no memory, any idea of "the conversation" or "the agent knows X" really means we fed that text back into the next call. Memory is something we manage in our code, not something the model has.
• Because the model cannot touch the world, it can only ever propose. Something in our code always decides and acts. That gap is where all our safety lives.

From a text function to an action: tools​

A text function cannot post a Gig::Shift. So we play a structured game with it. We give the model a menu of functions it is allowed to ask for. We describe each one as a schema (the function name plus its typed inputs):

You may ask to call any of these:
- resolve_pay_rate(outlet_id, role_id, starts_at)
- create_draft_shift(job_id, starts_at, ends_at, headcount)
- search_outlets(query)

The model cannot run these. It can only emit a message that says "I want to call resolve_pay_rate with outlet_id: 42, role_id: 7, starts_at: 2026-07-05T09:00+08:00".

Our Rails code then runs the real method. We take the result and feed it back into the next model call as more text.

The model never touches Gig::PayRate. It never walks our fallback chain. It only asks. Our own tested method does the real work and returns a fact.

A "tool" is not a new capability. A tool is a service object we already have, wrapped with a schema that describes its inputs. resolve_pay_rate is our existing Gig::PayRate.resolve!. create_draft_shift is our existing shift manager. We are not giving the model new powers. We are giving a text-driven caller a way to ask for powers we already built.

The authority boundary — the most important idea for us​

Read this section twice. The line between "the model decides" and "our code acts" is called the authority boundary (the point where a proposal from the model becomes a real action in our system).

Our invariants live on our side of that line. The invariant is the business rule that must always hold — for example:

• publishing a shift reserves gig credits so an employer cannot overspend (headcount × duration × rate),
• the suspension tiers escalate Warning → 7d → 30d → 90d,
• billable time locks at 9:00am the next day,
• a Gig::Job moves :active → :archived, never a hard delete.

These rules live in Rails, and the tool itself enforces them. The model proposes; our domain decides.

This is why a model that is confident but wrong is still safe. Say the model asks to create a shift with a rate it made up. Gig::PayRate.resolve! ignores the made-up number and resolves the correct rate from the fallback chain. The tool validates. The model cannot break an invariant by sounding sure.

What an "agent" really is: a loop​

Once the model can ask for tools and read results, we can put it in a loop. This loop, about twelve lines, is the whole thing. There is no magic.

messages = [system_prompt, user_goal]

loop do
  reply = model.call(messages, tools: TOOL_SCHEMAS)

  break if reply.done?                   # the model says the task is complete

  result = execute_tool(reply.tool_call) # OUR code runs the real method
  messages << reply                      # remember what the model asked
  messages << result                     # remember what actually happened
end

The loop is: the model picks an action, our code runs it, the result goes back, repeat until the model says it is done. That loop, plus a set of tools, plus a goal, is an agent.

Here is a real shape for Jod. Imagine an "employer assistant" where a FairPrice manager types:

"I need 3 servers at the Orchard outlet this Saturday, 6pm to midnight."

The agent loops:

1. asks to call search_outlets("Orchard") and gets the outlet_id
2. asks to call resolve_pay_rate(outlet, "server", saturday_6pm) and gets the weekend night-band rate
3. asks to call create_draft_shift(...) and gets a draft back
4. replies: "Draft created, 3 servers, Saturday 18:00 to 00:00, rate $X/hr. Please review and publish."

The model chose that order. Nobody hard-coded "first search, then resolve the rate, then draft". That runtime choice of steps is the one thing that makes it an agent.

Workflow vs agent — the decision we will keep making​

This is the fork we will stand at again and again. The cleanest way to see it:

A workflow is normal Rails code where a few steps happen to call an LLM. We write the order. The model only handles the parts that need to read or write natural language.

An agent lets the model decide the order.

A simple test: can you draw the flowchart before it runs?

• If yes, it is a workflow. Encode the flowchart in Rails, and drop the model into the fuzzy boxes.
• If the flowchart depends on what the model finds as it goes, it is an agent.

Our default should be workflow-first. Most of the value we want — triage, extraction, classification, drafting — is workflows. Agents are a smaller and more expensive special case. A workflow fails in bounded ways we can test. An agent fails in open-ended ways: wrong tool, a loop that does not stop, drift away from the goal. For a domain that moves money and touches a worker's suspension record, bounded failure is worth a lot.

What the SDKs actually give us​

We could write that twelve-line loop ourselves. For a first demo we should. The SDKs (flueframework, LangGraph, RubyLLM, Raix, the Vercel AI SDK) earn their place on the boring, hard parts. They do for agents what Rails does for HTTP. We could write raw Rack, but the framework gives us conventions for:

1. The tool-calling loop — turning our Ruby methods into schemas and back, parsing the model's requests, running the loop.
2. Structured output with validation — "give me back {role, outlet, headcount, starts_at}" and guarantee it matches a schema, retrying the model if it does not. This turns unreliable text into typed data.
3. State and memory — what text gets carried into the next call, and how to shorten it when it grows too long.
4. Orchestration parts — chains, routers, running steps in parallel, and human-in-the-loop pauses (stop, wait for a human to approve, then continue).
5. Retries and fallbacks — model A fails, try model B; a tool raises, feed the error back so the model can recover.
6. Tracing — a full record of every prompt, tool call, and result for each run. In production this is the part we will care about most, because it answers "why did it do that?".
7. Streaming — sending output to our React screen token by token.

Four Jod examples, from least to most autonomy​

This ladder is the practical heart of the post. Same domain, four levels of autonomy. Notice the value is highest, and the risk lowest, at the bottom.

Level 0 — no LLM at all​

Pay-rate resolution. Walking the fallback chain is fixed business logic:

• outlet-specific time band →
• company-wide time band →
• outlet-specific day_type →
• company-wide day_type →
• default rate.

The day_type comes from Geo::PublicHoliday, weekend, or weekday. The input is structured. The rule is known. This is a plain Gig::PayRate.resolve! method.

An LLM here would be slower, cost money, and be wrong sometimes, for zero benefit. The first skill is knowing when not to reach for a model at all. If the input is structured and the rule is known, it is code.

Level 1 — workflow: a chain of fixed steps​

Suspension appeal review. A talent cancels a McDonald's shift 2 hours before it starts, gets a pending suspension, and uploads a medical certificate (MC) inside the 24-hour appeal window. The steps are fixed Rails code:

1. (Rails) guard: is this still inside the 24-hour appeal window?   <- invariant, never the model
2. (LLM)   read the uploaded document. is it a medical certificate?
           extract { issue_date, covered_dates, clinic }
3. (Rails) do covered_dates include the shift date?                 <- fixed check
4. (LLM)   draft a short, plain summary for the Ops reviewer
5. (Rails) put it in the queue for a human (Kimberly) to approve or reject

The model does only what it is good at: turning a photo or PDF into structured fields and a readable summary. Every rule that matters — the 24-hour window, the tier escalation, the final approve or reject — stays in Rails or with a human. This is a workflow, and this is where we should start.

Level 2 — workflow: routing​

Inbound message triage. Employers and talent send free-text messages. One model call reads the message and returns a label. A Rails case statement routes it:

• payment dispute → Finance
• cancellation emergency → Ops
• "how do I post a shift" → self-serve help

The skeleton is fixed. Only the classification step is fuzzy. Still a workflow.

Level 3 — agent: the model drives​

The employer assistant from earlier — post a shift by chatting. Here we do want the model to choose the order of tools, because a real conversation is open-ended ("actually make it 4 people, and what would Sunday cost?"). Tools: search_outlets, resolve_pay_rate, check_credit_balance, create_draft_shift.

One rule holds it together: create_draft_shift creates a draft, not a published shift. Publishing reserves credits, so a human clicks "publish". The agent gets freedom over the conversation. It never gets freedom over spending money.

Level 4 — high autonomy, where I would push back​

A tempting target: an "auto-staffing agent" that watches under-filled shifts and runs the selection on its own.

But our auto-selection is already a fixed, written rule (AM shift deadline 9:00am the previous day, PM shift deadline 5:00pm the previous day, skip suspended or overlapping talent, pick the top-ranked by our score: show-up 30%, cancellation history 25%, experience 25%, ratings 20%). That is a scheduled job, not an agent.

Handing ranking and selection — which directly creates Gig::Assignment and then Gig::Payment — to a model that is not deterministic trades a rule we can test for one we cannot. When the flowchart already exists, an agent is a step backward.

The checklist for deciding​

Six questions to run any proposal through:

1. Does this even need a model? Structured input and a known rule → code. Free text or a genuinely fuzzy judgment → a model earns its place. The pay-rate chain is code. "Is this cancellation a real emergency?" is fuzzy.
2. Workflow or agent? Can we draw the flowchart first? Then it is a workflow. This is the default.
3. Where is the authority boundary? Invariants live in Rails, enforced by the tool. The model proposes; our domain decides. A confident wrong answer must be unable to break a rule, because the tool will not let it.
4. How will we test it? We cannot assert_equal a prompt. The new discipline is an eval set (a labelled dataset of past cases — for example 200 past appeal documents tagged valid-MC / fake / not-relevant — that we score the model against). This is our CI for something that is not deterministic. We must budget time for it.
5. What is the cost and latency? Each tool step is one round-trip to the model, run one after another. An 8-step agent is 8 calls in a row: tokens and p95 latency add up, and the text we send grows every step, because re-sending the history is the model's only memory. Workflows are bounded and often cacheable. Open-ended agents are not.
6. Can we see what it did? In production, tracing every prompt, tool, and result is not optional. Without it, "why did it draft the wrong rate?" has no answer.

Where this lives in a Rails and React system​

The agent or workflow logic lives in the Rails backend, because that is where the tools live — the tools are our domain. React's job is thin: a chat screen, streaming the reply, and a screen where a human approves.

Do not put business-critical orchestration in the browser. The model's output is untrusted input, the same as any user form. We validate at the boundary and trust nothing that came from the model until a tool has checked it.

"RAG" is worth demystifying while we are here. Retrieval-Augmented Generation is just a tool that fetches relevant text and pastes it into the prompt. "Search the gig FAQ, take the top 3 chunks, put them in the prompt, then answer." It is one more tool, not a separate architecture. Do not let the acronym make it sound bigger than it is.

Why Jod is well placed for this​

Here is the part I find most interesting. The hard part of putting a model into a business is not the model or the framework. It is having a clean domain layer, with well-defined operations and clear invariants, to expose as tools. An agent on top of a messy codebase is dangerous, because there is no tool that safely enforces the rules.

Our doc-driven work is already that layer:

• our use cases (UC-1 publish, UC-3 auto-select, and the rest) are already named operations with preconditions → this is our tool catalog
• our model specs with explicit invariants (double handshake, settlement window, the three definitions of time, credit reservation) → these are our guardrails, the rules the tools must enforce so the model cannot break them
• our state machines → the legal transitions an agent is allowed to ask for

So the move is not "go build an autonomous agent". The move is: keep building the clean, invariant-rich domain layer we are already building. Then model features become a thin, low-risk layer of natural language over a core we can trust. The domain model is our real advantage. The framework on top is swappable plumbing.

Open questions to discuss later​

Things I want us to think through together:

• Which level-1 workflow should be our first real one? The suspension appeal review is a strong candidate — it is high-effort for Ops today, and the invariants are all on our side.
• Do we keep the agent layer in Ruby (tools = our service objects, in-process) or run a separate service? What do we lose either way?
• What would our first eval set look like, and who owns it? Without it we cannot safely change a prompt or a model.
• What is the smallest tracing setup that still lets us answer "why did it do that?" in production?

If any part of this is unclear on one read, tell me which paragraph, and I will rewrite it plainer.

Today ci:security failed (the step that runs npm audit).

• The real fix is one small change.
• But I saw a branch that "fixed" it by deleting package-lock.json and running npm install.

That makes the build pass, but it is the wrong fix and it is risky.

Two files, two different jobs​

package.json is the range we accept.

• When we write "react-router": "^7.12.0":
  • the ^ means: any 7.x version that is 7.12.0 or higher, but not 8.0.0.
  • so 7.13.1, 7.15.0, and 7.16.0 are all accepted.
• It does not name one version. It names a set of allowed versions.

package-lock.json is the exact versions we actually installed.

• Every package, direct and indirect, with its exact version number.
• We commit this file so that everyone installs the exact same code:
  • your laptop
  • my laptop
  • CI
  • production Docker
• npm ci (used in CI and in our Dockerfile) will not even run without it.

The rule that explains everything​

When package-lock.json exists, npm install and npm ci install the version written in the lock file — not the newest version in the range — as long as the locked version is still allowed by package.json.

Read that again.

• ^7.12.0 does not mean "always give me the newest 7.x".
• It means "the newest 7.x is allowed".
• Allowed is not the same as chosen.
• Once the lock file exists, the lock file chooses.

So why were we stuck on the old version?​

• package.json says ^7.12.0.
• package-lock.json says 7.13.1.
• Is 7.13.1 still allowed by ^7.12.0? Yes.
• So npm keeps 7.13.1. Every install. Every deploy.

The fixed version 7.16.0 was already available the whole time, and it is also allowed by ^7.12.0.

• But npm never picks it, because a locked version is never upgraded on its own.
• That is the whole reason the build kept failing while the fix was already allowed.

Why npm audit fix did not help​

• npm audit fix did nothing.
• npm audit fix --force gave an error.

Why?

• The four @react-router/* packages must all be the exact same version as each other.
• npm audit fix cannot upgrade them together safely.
• That is why it felt like deleting the lock file was the only option. It was not.

What deleting package-lock.json actually did​

When you delete the lock file, npm has no recorded versions to follow.

• So npm chooses every version again, from the ranges.
• "Newest allowed" becomes 7.16.0 for react-router. Good — that part is fixed.

But you did not only upgrade react-router:

• npm chose a new version for every package in the project — hundreds of them.
• Each one jumped to the newest version its own range allows.
• You wanted to change one thing, and you changed everything.

The result:

• The diff is huge.
• The one change you wanted is hidden inside hundreds you did not ask for.
• Any one of those changes you did not notice can break production.
• You do not know what you installed.

The correct fix — change only the packages you mean to change​

npm install \
  react-router@^7.16.0 \
  @react-router/node@^7.16.0 \
  @react-router/serve@^7.16.0 \
  @react-router/dev@^7.16.0

This does two things, and nothing else:

1. It updates the range in package.json to ^7.16.0, so we never allow the old version again.
2. It updates only those four packages in the lock file. Everything else stays exactly where it was.

Now:

• the diff is four lines
• the build installs the same code everywhere
• a reviewer can actually read it

The principle​

package-lock.json is the file that makes "it works" mean "it works the same everywhere".

• It is not there to annoy you. It is doing an important job.
• When one package is stuck, change that package on purpose.
• Do not delete the record and reinstall everything from zero.

jangan hapus package-lock.json hanya untuk memperbaiki satu paket.

Intro to Json Shapes​

We will consider simple

GET /identities/user/:id​

For a simple endpoint like getting a single Identities::User, we would expect the response to be:

Response:

{
  id: ..
  country_id: ..
  email: ..
  first_name: ..
  last_name: ..
  phone_code: ..
  mobile: ..
}

GET /org/user-profiles/:id​

We explore the difference cases of considering associations when getting a single Org::UserProfile.

Response if we do not join tables in the backend:

• simply rendering the attributes on the model Org::UserProfile

{
  id: ...
  title: ...
}

Response if I join Identities::User in the backend:

• Org::UserProfile belongs_to Identities::User

{
  id:  ..
  user: {
    id: ..
    first_name: ..
    last_name: ..
    ...
  }
}

Response if I join all associations of Org::UserProfile in the backend:

• Org::UserProfile belongs_to Identities::User
• Org::UserProfile belongs_to Company

{
  id:  ..
  user: {
    id: ...
    first_name:
    last_name:
    ...
  },
  company: {
    id: ...
  }
}

Bad response:

This is a badly designed response because there is no database backed model that follows the following format.

{
  id: Org::UserProfile.id,
  first_name: Identities::User.first_name,
  last_name: Identities::User.last_name,
  ...
}

The idea is to maintain the json "shape" to follow the backend models.

Problem Statement​

For every careers_job_application, I want to know who posted the careers_job

If I add another relationship to careers_job_applications, do I need to join all the tables?

Since AuthProvider wraps the entire frontend application

• AuthProvider stores Identities::User and Careers::UserProfile and Org::UserProfile (depending where the user logs into)
• Every page will know who is calling our backend API
• This means that the backend doesn't need to join the identities user

However, this only applies for only Career::UserProfile cases.

For an Org::UserProfile, they would need to know names of Careers::UserProfile

• For example, for each Careers::JobApplication, we would want to display the person name, which lives in Identities::User

GET /org-user/careers/job_applications​

• As a org user, I want to see who applied for my job

Looking at the relationship:

The person making this call is an org-user, I need to join the tables to display the name of the user who applied.

I would return the following response:

{
  careers_job_applications: [{
    id:,
    //.. attributes
    career_user_profile: {
      id: ..,
      identities_user: {
        id: ..
        first_name: ..
        last_name: ...
      }
    }
  }]
}

GET /org-user/careers/job_applications/:id (#show)​

When showing a single job application:

{
  careers_job_applications: {
    id:,
    //.. attributes
    career_user_profile: {
      id:           ..,
      certificates: [{ .. }],
      experiences:  [{ .. }],
      skills:       [{ .. }],
      identities_user: {
        id:         ..
        first_name: ..
        last_name:  ..
      }
    }
  }
}

How can we reduce joins to improve performance?​

Add a json column career_user_profiles.data, every time we save a Career::UserSomething (i.e. skills, experiences, certificates), we update two tables:

• Career::UserSomething table
• Career::UserProfile.data columns

{
  careers_job_applications: [{
    id:,
    //.. attributes
    career_user_profile: {
      id: ..,
      data: { // new column that pre-computes the "join" of the associated tables
        certificates: { .. },
        experience: { .. },
      }
      identities_user: {
          id: ..
          first_name: ..
          last_name: ...
        }
    }
  }]
}

The downside of this is that we require to update the tables whenever any one of the Career::UserProfile associations change.

• then again, it's fairly simple to ensure this by ensuring we call a, for example, Careers::UserProfiles::UpdateDataService in all the places that updates Career::UserProfile associations.

Charlie Question

Imagine we are in a company backend and frontend are two separate teams.

• I can see that response will always follow the convention where the payload will be correctly formatted (i.e. following the model "shape")
• can frontend user ignore the shape when they are submitting the form?
  • I imagine frontend does not have visibility how it is implement in the backend

Backend creates endpoint:

• Send the postman doc to Frontend
• Frontend would see it and implement their designs
• Frontend realise in a particular page UI design, the endpoint provided does not return data from a specific model
• Backend say i cannot join tables any more cause the response you got initially is already joining 10 tables
• Backend will say, you call this other endpoint to get that model

So what have we established?​

• The default API Response JSON "shape" should follow the model associations.
• Ideally max up to 3 table joins in the backend

How does this "model association json shape" affect forms in the frontend?​

Forms should always follow the model it is rendering for.

If want to create a Car, I would have a CarForm that follows the Car attributes.

• one form field for each Car attribute.

Likewise, if want to create a Org::UserProfile, I would have a OrgUserProfileForm that follows the Org::UserProfile attributes.

Org::UserProfile Attributes

• id
• identities_user (association)
• company (association)
• title
• created_at
• updated_at

In my <OrgUserProfileForm>, I would need fields for the associations as well.

Scenario 1: Admin​

Assume Admin (jod staff) is creating an Org::UserProfile.

• Admin can access everything in the system
  • they can see all identities_users and companies

<Form>
  <TextInputField name={'title'}>
  <IdentitiesUserSelectField name={'identities_user.id'}>
  <CompanySelectField name={'company'}>
</Form>

const IdentitiesUserSelectField = () => {
  // makes an api call to GET /admin-user/identities/users
  // gets a list of identities
  // render the list of identities in a dropdown
  // user select 1 identity from the response
  // update the form OrgUserProfileForm state via react-hook-form
}

When I select a Identities::User and Company from their respective select fields

• my form state now has the Identities::User attributes and company attributes.
• this means I can nicely render the models int he frontend after the user selects.

Scenario 2: Org User​

Assume Org User is creating an Org::UserProfile.

• Org User can only access everything associated with their company
• They can only see Identities::User and a single Company, which is the company they belong to.

Thinking about the form schema for OrgUserProfileForm

const orgUserProfileSchema = () => {
  title: zod.string()
  // company_id: zod.integer()
  company: {
    id: zod.integer().required()
  }
  identities_user: {
    id: zod.integer()
  }
}

const OrgUserProfileForm = () => {
  return (
    <Form>
      <TextInputField name={'title'}/>
      <TextInputField name={'identities_user.first_name}/>
    <Form/>
  )
}

Yuri Question

Job Application <-- we need to insert this
- User Career Profiles  <-- we don't need to insert anything in this one
   - User Career Experience <-- we need to insert this

I want to create a Careers::JobApplication

• and at the same time create Career::UserExperience

Looking at the model associations (not the association on the database level)

TL;DR: Yes we follow the format.

// form schema follow "shape"
const jobApplicationSchema = {
  careers_job: {
    id: 1,
  },
  careers_user_profile: {
    id: 1 // we need to know who to create the Careers::UserExperience for
    careers_experience: [{
      job_title: ..
    }]
  },
}

// Alternative form schema which does not consider "model association json-shape"
// While it's not wrong from a technical perspective, this is not our convention.
const jobApplicationSchema = {
  careers_job_id:
  careers_user_profile_id:
  careers_experiences: [{ .. }, ...]
}

Imam Highlight:

Whenever you change any form schema/state shape, ensure you are assigning errors to the correct form field names.

• this will ensure that errors are rendered on the correct field in the form

Naming things​

July Direction​

Week 1 we will make changes to existing MVP based on the technical feedback from ali

• naming
• api layer should not have async
• form schema should follow endpoint structure/shape
• endpoint should follow shape of model associations

Week 2

• start on the peripheral services
  • send email
  • simple notification system
  • event/analytics
  • file upload

Week 3

• continue with peripherals
• start on product feedback

Week 4

• continue on product feedback

Docusaurus blogging features are powered by the blog plugin.

Here are a few tips you might find useful.

Blog posts support Docusaurus Markdown features, such as MDX.

This is the summary of a very long blog post,

Use a <!-- truncate --> comment to limit blog post size in the list view.

Lorem ipsum dolor sit amet...