Building an AI-Powered HRMS: Architecture Lessons from NexHR
- published
- updated
- reading time
- 4 min read
- topic
- AI / ML
NexHR is an HR system we designed and built at Bitlogs. It uses AI in four places: ranking resumes, recognising faces at check-in, writing candidate summaries and answering employee questions from company policy. Payroll sits next to all of that, and it uses no AI at all. The main thing we learned is to build each AI feature as its own piece that can be tested alone and has a manual backup, and to never let a model anywhere near salary numbers.
On this page · 6 sections
What did we actually build?
- Resume screening. We turn each job post and each CV into embeddings with BAAI/bge-base-en-v1.5 (768 dimensions) and store them in pgvector. JobBERT pulls the skills out of the CV text, and candidates are ranked with a formula HR can see.
- Face check-in. OpenCV finds the face, MobileFaceNet on ONNX Runtime recognises it, and a 100 m geofence confirms the person is at the office. If the camera struggles, employees can check in manually.
- Payroll. It collects attendance, approved leave, loan instalments and approved expenses, works out gross-to-net pay with fixed rules, and pays salaries in bulk through Stripe.
- HR assistant. Company policy documents are indexed in pgvector, and Google Gemini answers questions from them and shows where each answer came from. It can also look up employees and file leave requests, but only when the user is allowed to.
- Everything else. React and TypeScript on the frontend; Django REST Framework, Celery, Redis and PostgreSQL behind it. Login is JWT or Google Sign-In, and each company sets up its own roles (Admin, HR Manager, Finance Manager and Employee to start with).
How do you score resumes without hand-written rules?
Keyword matching works badly on resumes because people describe the same skill in twenty different ways. Embeddings handle that much better. We embed the job description and each candidate profile, then rank them with a weighted score: Final Score = (0.50 × semantic similarity) + (0.30 × skills coverage) + (0.20 × experience match). We kept the weights out in the open on purpose, so HR can see why someone ranked where they did and explain it to a hiring manager. That turned out to matter about as much as getting the ranking right.
Why run face recognition through ONNX?
Most people check in within a few minutes of each other at the start of a shift, so speed was the thing to get right. Early builds took over 3 seconds per check-in. We switched to MobileFaceNet on ONNX Runtime, compressed frames in the browser before upload, and added an OpenCV Haar cascade that throws out frames with no face before the heavier model runs. That got check-in under 1.5 seconds on ordinary CPUs. The 100-metre geofence means the right face in the wrong place still gets rejected. Just as important was the manual option: if bad lighting or a cheap webcam stops recognition from working, the employee can still check in.
Why keep payroll deterministic?
Payroll is where a lot of AI products cut corners: a nice dashboard on top, a spreadsheet underneath. We didn't want that. Before NexHR works out anyone's pay, it pulls their verified attendance, approved leave, any loan instalments they owe and their approved expense claims. Unexplained absences come off at the employee's daily rate, then tax and statutory deductions are applied. Run it twice and you get the same number both times. Payslips go out as PDFs by email, and salaries are paid in bulk through Stripe. A finance manager still checks and approves every run. Everything up to that point is calculated from live data.
What would we repeat on every AI product?
Frequently Asked Questions
No. We designed and built NexHR ourselves and held it to the same standard as client work: separate data for each company, role-based access, rule-based payroll and a manual option for every AI feature. It hasn't been rolled out to paying customers yet, so the numbers in this article come from our own testing, not from real customer traffic.
NexHR already runs on PostgreSQL, and an HR policy library is tiny by vector search standards. Keeping embeddings in pgvector gives us one database to back up, secure and query, and we can join policy chunks against normal tables like roles and departments. A separate vector database would be one more service to run, for very little benefit at this size.
The score is a simple weighted formula: 50% how closely the candidate matches the job description, 30% skills coverage and 20% experience. HR can see each part and adjust the weights. A model does the matching, but the ranking itself is basic arithmetic that anyone can check.
Payroll has to give the same answer every time and hold up in an audit. Language models don't guarantee that, so NexHR works out pay with ordinary rule-based code. The LLM is only used where a written answer with sources is what you want: the HR assistant and candidate summaries.