Looks like a good exercise. I have a few questions: 1. Redis is doing double duty as a blob store. I am curious as to what happens to memory, persistence, and failover as volume grows. 2. "Zero-copy" is loose wording because a copy into RAM rather than disk, not zero-copy in the strict sense. 3. I didn't see anywhere in the The README that mentions no retries, dead-letter queue, or idempotency and one of my biggest concerns is that the scaling policy is aggressive and possibly expensive because an A100 may be oversized for a 4B model. Finally, the license is inconsistent. The README says MIT, but the repo sidebar says Apache 2.0.
The GPU quota trap is the bit nobody warns you about until it costs you a day. I run sixty agents across an e-commerce stack and the lesson is always the same: the infrastructure that works on cloud A does not port to cloud B by changing the import statement. The model is the easy part. EFS throughput modes, node taints, KEDA scaling policies - that is where the actual engineering lives. Good on Christophe for documenting the traps instead of just the architecture diagram.
Apart from the learning experience, why is the infra so overcomplicated / expensive? Do you really need Kubernetes for this? In the current state, this is only somewhat better than Gemini 2.5 Flash, provided it runs 10 hours per day on workdays and the engineer is free. And that's millions of pages to OCR.
You certainly need a real challenge at scale (or deploy this architecture locally in your own GPUs) to motivate such deployment. This is the kind of architecture we build at @Credicorp to process +100k pdfs and +300k images on a monthly basis.
I understand this is not the kind of deployment, as it is, you would make as a freelance to process your own bills, unless you deploy it locally. But this production OCR course was never meant to target that kind of profile! This is an enterprise level solution :)
Appreciate the whole work ! 💯
@Ankit Singh absolutely, @Christophe Reigner has done a PHENOMENAL work here!
Looks like a good exercise. I have a few questions: 1. Redis is doing double duty as a blob store. I am curious as to what happens to memory, persistence, and failover as volume grows. 2. "Zero-copy" is loose wording because a copy into RAM rather than disk, not zero-copy in the strict sense. 3. I didn't see anywhere in the The README that mentions no retries, dead-letter queue, or idempotency and one of my biggest concerns is that the scaling policy is aggressive and possibly expensive because an A100 may be oversized for a 4B model. Finally, the license is inconsistent. The README says MIT, but the repo sidebar says Apache 2.0.
I had this on my mental backlog but you sure saved me a lot of time! great work!
Amazing work @Christophe Reigner
The GPU quota trap is the bit nobody warns you about until it costs you a day. I run sixty agents across an e-commerce stack and the lesson is always the same: the infrastructure that works on cloud A does not port to cloud B by changing the import statement. The model is the easy part. EFS throughput modes, node taints, KEDA scaling policies - that is where the actual engineering lives. Good on Christophe for documenting the traps instead of just the architecture diagram.
Apart from the learning experience, why is the infra so overcomplicated / expensive? Do you really need Kubernetes for this? In the current state, this is only somewhat better than Gemini 2.5 Flash, provided it runs 10 hours per day on workdays and the engineer is free. And that's millions of pages to OCR.
You certainly need a real challenge at scale (or deploy this architecture locally in your own GPUs) to motivate such deployment. This is the kind of architecture we build at @Credicorp to process +100k pdfs and +300k images on a monthly basis.
I understand this is not the kind of deployment, as it is, you would make as a freelance to process your own bills, unless you deploy it locally. But this production OCR course was never meant to target that kind of profile! This is an enterprise level solution :)