When Frontier Models Are Overkill With Javier Fernandez Rico
In this episode of AI Explained, we are joined by Javier Fernandez Rico, Head of Engineering at Fastino Labs, where Fastino builds small, task-specific models like GLiNER and the GLiDE decision model, plus an agent that helps teams fine-tune them. Javier has spent his career fitting AI into tight constraints, from building the first AI team at PlayStation for PSVR head, eye, and hand tracking to building simulation environments at Cruise.
He and Josh Rubin, Head of AI Science at Fiddler, dig into why teams default to frontier models and when that stops making sense at scale. Javier explains why 500 to 1,000 clean examples can be enough to fine-tune a small model, why a strong eval matters as much as clean data, and how chain of thought and LLM-as-a-judge can make simple classification less reliable. They break down decision models, which read the full input at once and return a calibrated score instead of generating tokens, and where they fit best, from guardrails to intent routing. They close with audience questions on LoRA vs. knowledge distillation and how to handle edge cases with monitoring, retraining, and model escalation.
Introductions
[00:00:05] Josh Rubin: So, uh, welcome, and thank you for joining us today for, uh, for AI Explained. So, uh, our topic is, uh, "When Frontier Models Are Overkill," and we're featuring our, our friends from Fastino. Um, I'm Josh Rubin. I lead AI science at Fiddler. Um, and I'm gonna be the host today. So we have a special guest today. It's, uh, Javier Fernandez Rico, um, who is the Head of Engineering at Fastino. Uh, welcome, Javi.
[00:00:33] Javi Fernandez Rico: Hi, Josh. Excited to be here
From Robotics to Small Models: Javi’s Background
[00:00:35] Josh Rubin: Super. Yeah, great to see you. Great to have you. Um, so I thought we would kick it off by just, like, asking you a little bit about your background because I think it's interesting and I think it speaks to the, the theme. Um, so, uh, why... So, so I understand that, that robotics is your background or, or so I just learned a little bit about you. Um, tell me about how building, you know, robots or, uh, you know, real world things shapes a little bit how you think about models and, like, the trade-offs that are involved in, um, AI models
[00:01:09] Javi Fernandez Rico: Yeah. Um, I've been in the AI space for a while, so I've seen all these different cycles coming in and out of different types of, uh, what we used to call machine learning to now AI. Um, but most of my career, I have been putting AI into, into different things. So I started PlayStation, where I built the first AI team.
[00:01:32] Uh, we were building the PSVR, so the VR headset. Uh, and the VR headset needed to have some AI components, especially for the head tracking, for the eye tracking, for the hand tracking, um, where the model needs to fit, like, milliseconds inside a constrained console hardware or otherwise, like, the user can get sick, right?
[00:01:55] From there, I moved to autonomy, where I spent substantial amount of time at Cruise, um, the self-driving car company now acquired by GM, and that was kind of like building simulation environments to help kind of the cars predict what human behavior is gonna happen out of distribution and generate datasets out of it.
[00:02:17] Uh, then I moved to airplanes, where it was like, okay, can we actually have an AI system that can control the entire airplane on its own using just cameras, similar to what Tesla is doing with cars, but now with, uh, with an airplane, and this was at Airbus and Acubed. Um, and you don't have a cloud there, right?
[00:02:34] So it's always constrained, either, either in the car or in the airplane. Uh, you're always constrained to just the hardware that can fit inside that, um, that vehicle. Uh, and so you don't start with, "Oh, what is the biggest model that I can do that will generalize at everything?" You start with a model that can do the job very, very well and do it in a superhuman capability, uh, instead of trying to do a very generalized model.
[00:03:02] So you start from the budget and the task, and then you build the smallest possible model that will do that task really well and reliably. Um, after that, I moved to Zirco. Uh, so I saw the other part of building a platform to train a model with, with a specific data, and what data actually is the big, uh, the big unlock of, uh, an AI system, right?
[00:03:27] Proprietary data or specific datasets allows the agents to actually, uh, or allows your model to actually, uh, perform better than any other model. So, uh, and now here at Festino, basically, we're building an entire application for you to train your models, uh, an agent that understands what you need to build, and it goes and builds it for you on top of the small models that we have been building because the narrative continues where a specific model tends to perform better in a particular task, and that's been the entire narrative of, of all my career.
[00:04:03] Josh Rubin: This is a very cool, cool career arc. I, I... But a lot of those sound like they were fun things to work on. Um, I'll just a-add that I, I have a PlayStation VR just off camera in a drawer.
[00:04:14] Javi Fernandez Rico: All right.
[00:04:14] Josh Rubin: a little dusty these days.
[00:04:16] Javi Fernandez Rico: Yeah
[00:04:17] Josh Rubin: uh, that was a very cool, a very cool device, so I'm impressed that you got to, got to work on that.
[00:04:22] Javi Fernandez Rico: Okay
Why Teams Default to Frontier Models
[00:04:22] Josh Rubin: What you guys made work on a, you know, very limited, very limited hardware. So it just speaks to the, you know, working with various trade-offs in, in constrained environments. Um, you know, it's, it's... So I, I guess, you know, hearing you talk about this, you know, I think we've sort of gotten as an industry into this head space of extremes, right?
[00:04:43] There... We, we tend to have very narrowly focused AI that's trained for a sp- very specific task, know, can run inexpensively, can run very fast or in a very constrained footprint. And then there's the whole other end of the spectrum, you know. Like you think about frontier models on, you know, LLMs or vision models that can, um, you know, they can give you, uh, relationship advice.
[00:05:07] They can help you, uh, you know, uh, find recipes for cooking dinner. They can, uh, help you, you know, write, write code, can help you solve technical problem. They're very extremely general, I mean, multi-modal, um, deep reasoning. Um, it, it's an interesting moment because I think people are starting to think about like what resides in the middle.
[00:05:29] How do you get some flexibility but, uh, you know, you know, uh, benefit from constraining tasks a little bit? So, so I don't know. I, I, I think a lot of teams are just defaulting to like the biggest models available. Um, I don't know. Where... Like where do you feel like that makes sense, and what do you think...
[00:05:49] where do you think the instance comes from?
[00:05:51] Javi Fernandez Rico: Yeah. Um, very opinionated, 'cause Fastino focuses on small models, so kind of biased towards that area. However, uh, it makes sense to default to the big models when there's a lot of unknowns, right? Like, it's the safe pick. Um, when a team doesn't really know what they need, when a team doesn't really want to explore the possibilities of owning their own system, uh, or they don't have the capability internally to build their own system, then defaulting to the biggest model that will basically do the reasoning for you and explore everything for you and pick the best solution for you is the right instinct, right?
[00:06:33] You're gonna start by basically giving everything to the biggest model, which to your point, it has a lot of capabilities of generalization and ability of exploring and now coding, and, um, as long as it's well contained in a sandbox environment, then you, you can do a ton of stuff there. However, over time, as you continue to build the system, right, um, that doesn't make a lot of sense 'cause it becomes economically unfeasible to continue.
[00:07:04] Uh, the, the cost of operating the biggest models is never meant to continue that way. That's why a- also the bigger labs offer three or four different variants of their models, right? Like the super extreme good model that is extremely expensive, like the Astras or the Fables of the world. Uh, and then you have, like, from there you go down the stack in smaller and smaller sizes.
[00:07:29] And as you explore what the needs of your program is or your pipeline is, it ... by default you're gonna go down that stack. So you're gonna start to filter out that big model into smaller one, and that's why we see all this, uh, pickup on decision models nowadays and stuff like that because it's just, like, a lot of people started with the big model, and then they start to realize, "Hey, I don't need that much reasoning for this part of the pipeline.
[00:07:53] I just need a very simple model that I can use Luna for, or I can use, um, a, a decision model for," um, and that's, that's what, that's what is happening. But the default is always the biggest one when you don't have all the answers or the expertise, uh, in that domain where you need a little bit of guidance.
[00:08:15] Josh Rubin: Yeah. Yeah, that's great. I think that, um, you know, for us at Fiddler, we think about, um, know, we kind of draw this distinction between, like, the difference between kind of prototyping something and then actually operating it at scale. So, you know, um, you know, you know, one reason I think that, you know, people often default to big models, and I, I, you know, I, I regard even a, a Luna or a, or a Haiku as a big model, uh, you know, sort of like big and all the way to enormous. Um, you know, is that, uh, you know, for prototyping purposes, you know, if you have some finite amount of data, you're working with fixed datasets, you can estimate the cost. you know, uh, it gets you wheels on the road for an evaluation, like an eval project, offline evals or, um, you know, experimentation really quickly, you know, with a lot of...
[00:09:10] without a lot of, you know, finickiness. but that, you know, the thing kind of blows up once you're at scale, when you're talking about, um, either guardrails or, um, you know, near online evaluation observability when you might start looking at like, you know, ten thousand or a hundred thousand, you know, uh, inputs per day.
[00:09:31] All of a sudden, even on the smallest frontier models, you're, you're, you're paying for a lot of, um, general capability that isn't typically necessary for the narrow task you're trying to evaluate. now, you know, I don't think that every org wants to go down the rabbit hole of like, you know, fine-tuning tiny classifiers on every task that they want to do.
[00:09:53] And I think that's actually where, um, you know, Fastino's stuff, whether it's like, you know, GLiNER or GLiDE, you know, decision models or, I don't know, you can, you can, we can maybe a little bit later speak to like the, the spectrum of, um, offerings you guys have. But, um, you know, I don't know if you want to speak to like... I, I think there's a really nice kind of middle section of this, of the size and cost spectrum where, um, you know, these narrower models can fit and still provide a lot of like flexibility. You know, you don't necessarily have to go to this like, um, special purpose fine-tune thing. So I, I don't know. Do you...
Fastino’s Place in the Model Ecosystem
[00:10:30] Josh Rubin: how do you, how do you think about like Fastino's offerings and like where it fits into the ecosystem?
[00:10:36] Javi Fernandez Rico: Totally, yeah. Uh, yeah, Facino historically has always been offering, like, very small, what we consider small models to, like, small brushing medium-sized models. Um, and, and we offer this for a reason because, uh, to your point, right? Like, you need to have this spectrum of options depending on the problem that you're trying to solve.
[00:11:00] But the reality is that within this space, which spans from, like, 200 million parameters that can run on your laptop all the way to, I don't know, 27 billion or something like that parameters, which is nowadays considered on the smaller spectrum You, you have plenty of tools at your disposal to actually, uh, fine-tune that particular model without the necessarily go down to, oh, this is just simple machine learning model that it's super tiny, very specific for this task.
[00:11:31] You s- you have an, uh, an option for generalization where, for example, our model GLiDE is a very popular model nowadays. It's a decision model that it's ranking number one in different benchmarks. So out of the box, it's very capable, right? Very generalized capability for making decisions on, on different, uh, complex topics and even multimodal capabilities where you can just, like, throw an image and get a decision out of it.
[00:11:59] However, like we've seen that if you own the data, if you have a little bit more insights of what exactly your pipeline requires, a little bit of fine-tuning on top of it will give you even bigger capabilities. So in some of the, uh, actual tests that we have benchmark against, we have beat Opus 5.5 or, um, other similar big models, in particular tasks in a smaller footprint model without the need, the need of too much fine-tuning, right?
[00:12:35] So if you want to actually own that intelligence, if you own the data, you don't need to go, like, all the way to the extreme of c- of keeping the big model at all times. And even if, if you believe that your company doesn't need the fine-tuning, like fine-tuning now it's, uh, uh, we, we are making it very simple.
[00:12:55] But, but, uh, the reality is that even out of the box, these smaller models are pretty capable already. So not necessarily you need to always default by the biggest model. Like now with the capabilities of smaller models, and we're seeing a lot of other, um, smaller models popping up that are very, very capable.
[00:13:13] Maybe starting with small, it's the new default, right? Where I'm gonna try what the capabilities of the small model is first. If it doesn't pan out, then I'm gonna scale up. And it's similar to the approach that I talked, I took in my entire career, right? Where, um, you have like highly specialized tasks that you need to perform, and you start with small well-contained one.
[00:13:35] Maybe we'll, we'll start to see that trend, and hopefully we'll start to see it where you're not wasting a lot of computer necessarily because you can start by a strong eval and then see how the small model performs instead of just defaulting to the, to the big one, uh, all the time.
Finding the Right Model Size
[00:13:52] Josh Rubin: So do you think 27B is the sweet spot? I mean, it's where you guys have started with GLiDE, but I get the impression that you're, uh, that there's, that there are a l- a line of models that you have in mind. Do you think that the model size ends up being a, a selectable dimension?
[00:14:04] Javi Fernandez Rico: Yeah. I think 27B has been proven for several years now or months now, uh, years in AI space, I guess, that,
[00:14:14] Josh Rubin: Yeah
[00:14:14] Javi Fernandez Rico: that, uh, that it's, uh, that it's pretty capable in terms of, like, performing a, a generalized, in a generalized way where it's performing relatively well in coding task, high thinking tasks, but also performing very, very well in specific tasks.
[00:14:31] So I would say 27B seems like a very good sweet spot for now. As we progress, right, I believe that we'll see even lower, uh, parameter counts performing really, really well. I think the trend is moving in that direction where 27B right now is a good spot, but we're gonna start to see even lower, lower and lower and lower parameter counts that perform really, really well
[00:14:56] Josh Rubin: Yeah, I was going to say, um, you know, we've done work on, um, on eight billion parameter models and found that, you know, if the task is sufficiently narrow and, uh, you know, you have good training examples, I, I guess this is cliché by now, but like uh, but, but, you know, quality of data is, uh, it's, it's remarkable how much that matters compared to everything else. Um, but what we found is that on an eight billion parameter model and a sufficiently narrow task and good data, you know, we're seeing eight billion parameter models with like LoRA fine-tuning, um, know, compete impressively against the smaller frontier models, you
[00:15:38] Javi Fernandez Rico: Yeah
[00:15:38] Josh Rubin: who, uh... It, it, it really speaks to this story about like, know, narrowing the task and you don't, for lots of things, need that like broad, um, domain generalization, uh, and how repurposable that, uh, that, that intelligence is, uh, if you can kind of bring it into a, a narrow task scope. Um, I've been sup- super impressed and, you know, the more that that becomes like a easy-to-use mechanism, I think it sounds like you guys are excited about, you know, what you're offering your customers in terms of, um, tunability. But, you know, when that becomes really turnkey, um, I think that makes a lot of things possible, uh, that, uh, you know, otherwise were kind of, um, hard to make work well.
[00:16:23] Took, you know, years of research or, you know, all the, all the blood, sweat, and tears that have gone into some of your earlier, earlier career steps.
[00:16:29] Javi Fernandez Rico: Yes.
[00:16:30] Josh Rubin: how much, like
[00:16:31] Javi Fernandez Rico: Yeah
How Much Fine-Tuning Data Do You Need?
[00:16:31] Josh Rubin: what is it, how much data do you ask your customers to bring to get some meaningful output from,
[00:16:37] Javi Fernandez Rico: Yeah. Uh, we don't need that much data, to be honest. Like, um, and to your point on, on the eight billion parameters, right? Like, our GLiDE model, our GLiNER model is extremely small. It's, like, 300 million parameters, and that fine-tuned performs really, really well for entity extraction, name recognition, PII tasks, all of those, uh, tasks that are very still needed across many industries.
[00:17:01] But we also have seen that, to your point, if you have, like, very good subset of data, you don't need that many, right? We've seen between 500 to 1,000 examples, you have plenty of, of, of data enough for a fine-tuning job, and we've seen, uh, a model smaller than 27B, like 8B, 4B, 2B, right? Like, perform, like, state-of-the-art because you're, you have, like, clean data.
[00:17:28] So we've seen between 500 to 1,000, it's more than enough examples to cert... Depends, of course, on the task, but to certain tasks it's more than enough. The more complex the task, you probably need a little bit more. We also kind of augment that data with synthetic data. So once you start with a good foundation of clean data, augmented works very, very well, uh, instead of generating it from zero, right?
[00:17:52] Like, if you don't have any examples and you just go, like, "Hey, prompting my way to synthetically generate data," it's not gonna work very well. But if you have a solid foundation of, I don't know, 1,000 examples, then y- you're gonna see that augmented data set, um, it's gonna perform way better. And then with, with only those, you're gonna get pretty good performance out of a very small model, um, for a particular task.
Building Evals and Choosing When to Fine-Tune
[00:18:22] Josh Rubin: Yeah. Um, what do you... So like, you know, how do you think about deter- like, you know, measuring what can go wrong and performance and, like, selecting the right thing to do? Like, where is the, uh You know, for a company that's trying to, you know, deploy some efficient model, how, how do you think about like You know, do they just kind of plug into the, the prompted, the prompted version of something like GLiDE and then decide that the performance isn't quite where they want it to be, and then kind of bump it over to a, a task?
[00:19:01] And like, what is the... what... how do you see people,
[00:19:04] Javi Fernandez Rico: Mm-hmm.
[00:19:04] Josh Rubin: um, running evals to make those determinations? Does that require a science team on your, on your customer side
[00:19:10] Javi Fernandez Rico: Uh, so there's two ways of doing it. So to context, right? We have an agent that we use to basically collect, curate, and then run the fine ta- training pipelines for any of the base models that we'll have or any other open source models that our platform supports. Uh, the agent capabilities, uh, and this is where the reasoning of an agent becomes very important, allows you to basically identify from a clean data set, uh, what you can do to build a very robust eval.
[00:19:44] If you don't have an eval, right? Like a lo- a lot of the time companies do have evals already that exist in their pipelines, right? To your point, they were defaulting a lot to these, uh, bigger models for quite some time already. So you have the traces, you have the data already where you know what you can do to, to...
[00:20:02] what, what works and what doesn't on that pipeline. So you can now, with all that data that you have collected over time, you can build an eval. But let's say you, you actually started from scratch. You don't have anything. Um, the data that you already own is very rich enough for you to construct a, a, a good eval, um, to test the performance of this particular model in that pipeline.
[00:20:26] There's also benchmarks that you can use, uh, that exist out there, um, or a complement of both, right? Like what we've seen so far is our agents tends to choose between like... They, they tend to blend between like an internal eval and a benchmark just to complement the scoring of that particular pipeline and get a more robust signal out of, of what they're building.
[00:20:53] So I would say start with, just as you were saying, like hand-curated data, very important, right? Clean data. I would say like as important, it's a very good eval . You know? Like if you don't have a very good eval, if you don't know what you're measuring against, there's no way you can have a good pipeline, even if it's...
[00:21:13] Forget about AI, right? Like you need to know like what you're expecting out of even heuristics, uh, or a normal coding as we used to do before, right? You need to have those evals, and in AI systems, they become way more important, especially if you're relying on LLMs where the under deterministic output can change over time.
[00:21:33] Um, if you move to models like GLiDE or GLiNER, we are more deterministic, which is a, a huge advantage of using models like this, where determinism is ingrained in the way the model behaves. Um- But each model has its own, uh, as we've been discussing throughout this conversation, each model has its own place and time, right?
[00:21:54] So you need to have those evals that will give you that signal, and that's why I think it's important to, um, experiment with the evals and see where the models that you're testing actually sit, and then make that decision of, hey, this base model is pretty good. Let's now move to the next stage, which is let's fine-tune it and see how much we can push that performance
Failure Modes: Overthinking and LLM-as-a-Judge
[00:22:17] Josh Rubin: What about failure modes of like general models? I don't know if you... I have some feelings about that I might, I might share, but I'm curious if, if, uh, you know, I, I guess I find that, um, I'm often surprised that if you go even to the large frontier models that for classification tasks, can often get really hung up on, um- It's almost like they're sort of, uh, over-fitting in their decision process and they can end up kind of leading themselves astray. They kind of overthink things that should be, you know, to a human, a fairly reasonable, uh, conclusion by getting hung up on like kind of nitpicky details. Like they don't quite know at what level to like, to like let it go, right? Um, and, and, and, and, and I found, you know, in, in our work that, that You know, if you want a really, like, high-quality classifier, for example, that, you know, you might find that, uh, you know, a really, a really large frontier model kind of poops out in the kind of low to mid 90% of accuracy or pick your classifier performance.
[00:23:28] What's I'm curious if you have other observations or feelings about like, you know, some of the perils of kind of overgeneralization
[00:23:38] Javi Fernandez Rico: Yeah, yeah. I, I think it, it's very fascinating to actually see what LLMs can do and what cannot. Cl- for simple task-like classifications, we've seen that they, to your point, tend to not be as reliable from time to time. Uh, it's probably the undeterministic behavior where you can send the exact same classify request and then, like, have, like, different results.
[00:24:03] We spent substantial amount of time, uh, in the past months on relying a lot on LLM-as-a-judge because of the undeterministic behavior where you're like, "Oh, this output, I don't know if I can trust it. Send it to another LLM to tell me if I can trust it," which is kind of, like, weird because that LLM will have the, the, the exact same problems ingrained.
[00:24:24] Uh, we have also seen that chain of thought tends to confuse them. So
[00:24:29] Josh Rubin: Mm-hmm.
[00:24:29] Javi Fernandez Rico: it's quite eerie because it's kind of like a human of some sort, where it's like if you're overthinking your stuff, sometimes you just ca- come out with a worse possible answer than just, like, going with your instinct. And, uh, we've seen, and I think that I, I read some papers around, uh, how chain of thought for simple classification tasks or tasks like those, uh, could affect the actual outcome in a, in a worse way, uh, than just going with a smaller model that is really good at just classifying or making a decision, right?
[00:25:06] And that's why we're seeing all this, uh, boom in classifiers, decision models, because they don't have these issues.
Synthetic Data and Production Feedback Loops
[00:25:14] Josh Rubin: Yeah, I think that, I think that there's something about the reasoning trace that, that is definite- Like, like I think, uh, you know, uh, as, as many labs have done, you know, we've often tried to use and, and effectively use like panels of LLM judges across a bunch of different big models. Uh, it's remarkable what they disagree on.
[00:25:32] And, you know, you store all their kind of thinking traces and it's like, they've all fixated on some weird, you know, uh, some weird detail that's, that's not totally relevant. But, you know, they're never quite sure how like nitpicky to be, uh, uh, which, which, which is awfully interesting. Um I think, so you, you mentioned just, just, uh, before, I, I, I want to know kind of more about decision models, and maybe you can... We'll, we'll get to that in just a second. But I think that, um, something else you mentioned that was interesting was sort of using prompts as a way to maybe generate data or fine-tune. I think you mentioned something like that. I, and I, and I know from our own work, we're heavily involved in sort of synthetic data, so oftentimes we don't have a huge number of real examples.
[00:26:23] For example, in, um, you know, kind of, um, the kind of cybersecurity threat detection, you know, oftentimes if you have a s- a dataset, there are, you know, times as many, you know, kind of benign examples as there are actual threats. And so you don't have, like, exploration of the, um, you know, the domain of possible threats and variations of threats.
[00:26:49] And so, you know, the kind of thing you might do, you know, we, you know, there's, there's training on real data that happened in the wild. There's training on, like, some hypothetical examples that you give into AI. I think, you know, we've landed in a place where, um, you know, we try to ground our synthetic data generation in sort of hard negatives.
[00:27:09] Like, basically, like find where the model is struggling
[00:27:12] and, uh, you know, try to drive, uh, you know, generate synthetic variations of those examples to provide a little more support for the model. I- is that part of, you know, the Fastino thinking about you use real data or bringing in synthetic data or, uh,
[00:27:31] Javi Fernandez Rico: Yeah
[00:27:31] Josh Rubin: tell, tell me about what your, your, your kind of workflow looks like.
[00:27:34] Javi Fernandez Rico: Totally. Uh, so once you have a model in production, right, it's not gonna be, uh, there's gonna be out of distribution events, as you're saying, where, "Hey, I've never seen this." It's just making a call, a prediction, and it might be the wrong prediction. Uh, we, we do monitor the traces of the models that we have fine-tuned, and over time we identify, we basically cluster pockets of where the failure modes happen, and then we, uh, generate more, more data in those failure modes.
[00:28:13] Um, most of it tends to be synthetic, 'cause to your point, there's not, there's no data. Otherwise, we will have trained for it. So we, we grab those failure pockets, and then we use the few examples out of the traces that we've seen from the, uh, the in-production model, and then we expand and, and synthetically generate more examples similar to that one.
[00:28:39] Uh, because it's in production though, you do get the traces, so you have an actual example. Uh so you tend to start to get some examples. It's not starting everything from zero. Now, if you have the expertise in-house, uh, I still believe that someone that knows what they're doing can actually steer a generator into creating data that actually works.
[00:29:05] Uh, but you need to have the expertise in-house of someone. In your, in your example, right, like you need to have someone that's very good at identifying security vulnerabilities and they make sure that it actually can generate examples like that. There's the other part of generating synthetic data, whereas a lot of synthetic data can be generated and verified by code, uh, in a lot of these examples.
[00:29:27] So, um, that is something that I always tell people to do a lot is like, "Hey, you can generate the data, run it, it should actually be verifiable." And we've seen that verifiable rewards actually work very well. Beyond just like reinforcement learning, it's a, it's an approach to actually generating synthetic data as well, because you can verify, hey, like this actual piece of code works, doesn't work, or it actually pro-
[00:29:51] Josh Rubin: that's for things like coding agents where you're building
[00:29:54] Javi Fernandez Rico: Yeah
[00:29:54] Josh Rubin: structured, meaningful data that can be
[00:29:57] tested.
[00:29:58] Javi Fernandez Rico: But
[00:29:58] Josh Rubin: it
[00:29:58] Javi Fernandez Rico: it could be beyond other stuff, right? Like, it could be, um, like math problems, or it could be physics, or it could be, like, things that you can codify and, and actually re- solve through code. Uh, it's something that you can create data and actually verify its outcomes, um, because you know, like, what the possible answers should be.
[00:30:20] And that is, like, more trustable synthetic data generator, but yeah, it, it will not work with domains that you can't... Like, I cannot do that with a legal document right now or something like that
What Is a Decision Model?
[00:30:32] Josh Rubin: yeah, yeah. Um, so let's... Maybe we can talk a little about decision models. So we've, um, you know, you were mentioning they're, you know, sort of more, more, more deterministic. Um, I don't know if you wanna... So I think, you know, in the last, you know, we've seen, um, you know, you guys have GLiDE. I think Jev hopped on the scene with a whole lot of fanfare, you know, whatever, six weeks ago, something like that, two months ago. uh, do, I don't know, can you, can you kind of frame for our, uh, listeners what, uh, you know, like what a decision model is and how it's different? Maybe we can get a little technical here if that's, uh...
[00:31:10] Javi Fernandez Rico: Yeah.
[00:31:10] Josh Rubin: I,
[00:31:11] Javi Fernandez Rico: Totally
[00:31:12] Josh Rubin: to me to... I, 'cause I think it has a... I think it's, there's some old, old, you know, well-established technology under the hood, but it's the form factor, uh, that is of novel and pretty inspired
[00:31:26] Javi Fernandez Rico: Yeah. Uh, let's just start with how, like, the LLMs work, right? Like, the LLMs just have, like, work by generating tokens one at a time, right? You're just basically trying to predict what the ne- the best possible next token is, and every token has a chance of divergence or wonder. Um, so that, that causes these undeterministic behavior, and that's why you see a lot of researchers saying like, "Hey," like, uh, "undeterminism is not a feat- is not a bug, it's a feature.
[00:31:57] That's what LLMs do. That's how they explore. That's how they generate." Uh, when you talk about decision models, a decision models reads the input all at once, and then it returns a score, uh, and it, it, it, it... You know the potential outputs, uh, for that particular query. So in, in that sense, you just get an actual calibrated score for a decision that, that's why they, we call them decision models, that you want to basically choose from.
[00:32:30] So it's, uh, it's less than a, uh, generate, uh, this open-ended question for me and more about, like, from these options, what is the best option, you know, out of this input? And that is very different, and it's very close related to just classifiers, where you're just classifying the best potential, uh, outcome, uh, or the best potential answer.
[00:32:54] But in this case, you have multiple choice answers. You can have binary answers, but you have multiple choice answers, and, uh, that's, that's the main difference. And why decisions model have been picking up is because you're removing all that undeterminism out of the answer, and it can actually be plugged into a coding pipeli- a code pipeline that you already have in your system and then just rely on the actual, uh, probabilistic result to pick the best answer and then drive with it.
[00:33:22] And you'll have a higher confidence that that answer is accurate rather than trying to figure out, like, out of this text and this JSON parsed output from the LLM, is this correct or not? Like, you move from is the actual output of the LLM not only accurate and good, but it is also parsable. Is there any bug there or something to, okay, the output is completely structured now.
[00:33:50] I don't need to worry about it. I just need to select the highest probability outcome and then just run with it. And of course, it can fail. That's where fine-tuning, uh, comes into play, uh, but that's, that's the main difference between LLMs decision models.
[00:34:06] Josh Rubin: That's great. Yeah, I think, uh, for anybody who's really wrestled with like s- like decoding, uh, you know, uh, generative, you know, uh, generative langua- language models, uh, you know, and expecting some sort of like reliability out of it, it's actually a really hard problem. You know, there's a lot of options, structured decoding of various kinds that turn out that they have a weird influence on, uh, the quality of the answer you get.
[00:34:31] As soon as you start, like constraining what tokens are available for the LLM to use,
[00:34:36] there's some huge rabbit holes in terms of like changing their behavior. As soon as you say, you know, your options right now are like s- a comma, a space, or a square bracket, uh, it, it sort of throws the whole algorithm off in, in, in interesting ways. I, I
[00:34:51] Javi Fernandez Rico: Yeah.
[00:34:51] Josh Rubin: won't go into war stories about that, but
[00:34:53] Javi Fernandez Rico: Yeah, the amount of, the amount of times
[00:34:56] Josh Rubin: just trying to understand, um, how best to use, you know... I think, I think as a, as a concept decision models
[00:35:03] Javi Fernandez Rico: Yeah.
[00:35:03] Josh Rubin: much cleaner and coupled to, um, deterministic code in a, in a much nicer way
[00:35:08] Javi Fernandez Rico: Yeah, the amount of times that I have spent on making my parsing, uh, code block work with an LLM output, I, I don't want to go in the horror story.
[00:35:21] Josh Rubin: It's crazy. Like,
[00:35:22] Javi Fernandez Rico: So
[00:35:22] Josh Rubin: there's, there's so many... I'm, I'm g- okay, I'm gonna steal
[00:35:25] Javi Fernandez Rico: Jayeeta
[00:35:27] Josh Rubin: But like, I mean, there are so many weird ways that it goes wrong, and even when it's operating properly deterministically,
[00:35:35] Javi Fernandez Rico: Yeah
[00:35:35] Josh Rubin: know, there are these, like, knock-on effects on the model's classification performance.
[00:35:40] So just, just, just crazy, crazy weird stuff that happens. We s- we, you know, uh, been playing with, um, you know, you know, within Fiddler for our evaluators, we provide some prompt ability and some evaluators that are based on, um, you know, small size generative language models. And, uh, you know, you sort of would expect to just be able to swap a new one in that tends to perform better. But there's these bizarre things like, uh, you know, if one model just happens to be more chatty than another model. Like, some models like to talk. And so
[00:36:14] you have this weird thing where you're like, you have to worry about the coupling between, like, deterministic software and behavior, learned behavior of a model, uh, that has real impact on, like, serving infrastructure and, um, you know, how much it costs to operate the models.
[00:36:32] It's just a total, total, uh,
[00:36:34] Javi Fernandez Rico: Yeah.
[00:36:35] Josh Rubin: uh,
[00:36:35] Javi Fernandez Rico: for
[00:36:36] Josh Rubin: total jumble
[00:36:37] Javi Fernandez Rico: and for automated pipelines, right, you don't care about what the model is saying. You just care about the output itself. So the chat becomes irrelevant at that point, and all of these LLMs have been just trained for, for chat output. So you're just wasting, you're just wasting tokens there.
[00:36:53] It's just like, "Hey, I just need the, the actual output. I don't-"
Decision Models: Cost, Speed, and Throughput
[00:36:57] Josh Rubin: you could also speak to, maybe speak to other trade-offs like cost and speed on that front. I mean, maybe that's, maybe that's implied, but, uh,
[00:37:03] I think that's another important point beyond the determinism is,
[00:37:07] Javi Fernandez Rico: Yeah. Yeah, I mean, uh, decision models are gonna be extremely fast, right? You're gonna get, like, on a fraction of, like, just in milliseconds, you're gonna get an answer. You don't need to wait for all the tokens to be generated. You don't need to wait for the chain of thought to complete. You don't need to wait for any of that.
[00:37:26] So latency becomes, like, very slow, very, very, very low. Um, especially if you are running it in your own infrastructure even lower, right? Like, a lot of, like, like, API calls going through the internet, like that adds latency, which is a detrimental to the performance of a decision model, which is inherently small.
[00:37:47] So if you actually run it, it's gonna be, like, extremely fast. You're, you're gonna not even, like, blink, and you will have an answer already executed. So that's one. The second one is, of course, the actual cost. Because they're faster, uh, because you need to process less, quote-unquote, tokens, uh, you're gonna need less hardware.
[00:38:09] Uh, they're gonna be inherently smaller, and then you're gonna run way more requests per h- per, per graphics card that you have. Uh, so you're gonna save a ton of money on, on actually running these decision queries rather than going and asking a big LLM to, to run this because they require multi GPUs. You require the decoder outputs.
[00:38:33] You require all those tokens to be generated. And then, uh, all that processing power goes to a final just quick decision task, which it's an overkill, so
[00:38:44] Josh Rubin: Yeah, yeah. I think, you know, one other aspect of that that I think is interesting is that it takes advantage of... Oh, look, we have a, a, uh, another poll question. you know, it takes advantage of the asymmetry between, um, input token and output to- input token, you know, prefill and output token generation.
[00:39:02] Javi Fernandez Rico: Yeah
[00:39:02] Josh Rubin: know, if you're not generating, you know, output to... I don't know what the actual, you know, typical number is, but it's probably... I think it's something like a factor of 10 in
[00:39:10] Javi Fernandez Rico: Yeah
[00:39:11] Josh Rubin: maybe in, uh, computational time to generate tokens versus, you know, populate with prefill and generate KV cache and that kind of stuff. And so, you know, if you can keep the answer short, it
[00:39:25] Javi Fernandez Rico: Yes
[00:39:26] Josh Rubin: of, uh, you know, a, a huge amount of compute, and it allows you to have low latencies and mu- much higher throughputs. Um, look at
[00:39:38] Javi Fernandez Rico: Yeah
[00:39:38] Josh Rubin: Uh, what, what, what matters most when choosing a model for production task? It looks like, looks like accuracy, uh, that's, that's interesting, is the, is the unanimous winner here. Um, but low, low stats on this one.
[00:39:51] Javi Fernandez Rico: I mean, of course, accuracy is gonna be the biggest factor, 'cause if you cannot actually have good answers out of the model, regardless of the size, the latency, the cost, right, then it's useless. I would say accuracy always is gonna be the biggest winner, uh, because if the model cannot predict properly what you want it to predict, then, uh, it doesn't matter.
[00:40:16] But after, after accuracy, I would say cost latency becomes very, very important depending on the, on the actual, um, on the actual industry that you're doing. But there's critical decisions that need to happen in a fraction of a, of a millisecond that you cannot wait for it to complete because it's a, it's a really big decision, right?
[00:40:36] Like let's take just self-driving car for a second, right? If you need to make a decision on, hey, like of course it needs to be high accuracy decision, but then the latency becomes extremely important of like a life or death decision there, right? Like I need to pick up like ... And I'm now gonna bring the trolley problem 'cause that was always brought up in self-driving cars.
[00:40:57] But just the simple decision of like, hey, should I just, uh, push the accelerator to go through the yellow light or should I just stop, right? Like you need a fast decision there because that will mean if you don't make a quick decision of whether you push through the yellow light or stop, you can cause an accident, right?
[00:41:14] So that is, um, that is, that is a, an example that comes to mind where definitely latency, uh, becomes very important.
[00:41:24] Josh Rubin: I th- I think interestingly on cost, latency, and accuracy, there's an argument to be made that for certain kinds of applications, um, decision models give you all three of those. It's... And I think, I think trying to s- to sort of put it up like a, a trade-off between those things is probably, incorrect.
[00:41:43] So I think, uh, one, one question I'd throw back to you is like,
When You Still Need an LLM
[00:41:47] Josh Rubin: what are we actually giving up, giving up? Like, like w- so maybe first like what are decision models like not well suited for? Like where do you actually want, where do you actually want LLM? And the one thing that comes to mind is, is like, you know, explanations or a self, self, you know, a reasoning trace or free response.
[00:42:07] But I, I'll, I'll let you, you respond to
[00:42:09] Javi Fernandez Rico: Yeah. I think, I think that the both models serve their own purpose, right? Decision models are fast decision-making. Uh, the LLMs still have their space when you need, like, exploration, explanation, uh, the actual thinking needed to reason about a problem, uh, that doesn't have a predefined answer, right? If I don't know what the outcome out of this, uh, input is, then I'm gonna rely on the LLM to go and explore, use coding capabilities, uh, use their knowledge that they already have, their reasoning capabilities to come up with a reasonable answer for me to actually use in, in my pipeline.
[00:42:48] Now, decision model, if you know potential outcomes, then doesn't, doesn't need the reasoning, right? It doesn't need the exploration. You know what the outcomes are if you're just picking between A, B, C, or D. Uh, and I think that's where the difference is probably biggest marked. Uh, now we're coming to the realization that the majority of the tasks in the world tend to be like a decision that you know the outcomes to, instead of a free exploration research type of task, uh, where you need this back and forward and chain of thought reasoning that the LLM is very good at.
Can Decision Models Act as Agents?
[00:43:23] Josh Rubin: This is, uh, so I'm gonna go off, off, totally off this here and, uh, if there's a flavor of decision models that are agentic decision models. Have you thought about
[00:43:35] like what, what, uh, like do you think you could create sort of a choose your own adventure kind of an agent that is a ... Like, so, so the thing that I think people point to that distinguishes agents from things like or specific tasks is, um, you know, that, uh, agents that kind of exist within some sort of a loop that allows them to, uh, choose what they need to know next and what to do and, uh, decide when they're done.
[00:44:03] Javi Fernandez Rico: Yeah
[00:44:04] Josh Rubin: think there's a crossover where there are decision models that behave like agents, or do you think they stay in this kind of narrow, like sort of guardrails, well-defined decision path, kind of fast, fast circuits?
[00:44:18] Javi Fernandez Rico: Um, I think there's an option there to make decision models in an agentic flavor. Of course, nowadays you have decision models that are in between the LLM and the, the, the agent and, uh, for example, things like tool calling or routing models and stuff like that. That's where decision models are very strong today for agentic systems.
[00:44:39] But, like, a lot of the performance gains that we've seen on bigger LLMs is the harness itself, right? That the harness plays a big part on how well Claude performs for coding or GPT performs for coding, right? Um, I think there's a universe that, uh, could exist where decision models, knowing the, the potential output could be put into a harness, basically a loop, um, where you can just let the a- the agent itself be the decision model and loop over time in different decisions.
[00:45:13] So you're sequencing decisions that will bring you a potential better outcome, um, because you have an entire pipeline of decisions that you can make. And we've seen it a little bit in people playing with the idea, like, um, there was, like, a ton of demos of, "Oh, yeah, I replaced Tesla Full Self-Driving with just a decision model."
[00:45:34] And it's essentially a harness that is just building on top of the previous decision, right, that the car to- took. So maybe for robotics, there's a good opportunity there. For smaller tasks where you have a, uh, uh, a, a good loop that you can build, where it's just decision over decision, the harness can be put in place, and you can have, uh, an, as you said, an in-between of an agentic system that just leverages decision models versus leveraging any LLM at all.
[00:46:03] So I, I think there's a potential there in sever- different applications that you could, you could do that without relying on the full LLM, because you don't, you don't require that reasoning and all of those, like, bells and whistles that come with big models.
Best Use Cases: Guardrails, Routing, and Observability
[00:46:16] Josh Rubin: Sure. So, uh, we have a few questions, but maybe, maybe one more from me before we get there. Uh, what, what do you think are the, like, the most killer use cases for decision models? I mean, maybe we can, maybe we can focus a little on like, know, for, uh, you know, supporting agents in some, some way. I don't know.
[00:46:36] What do you, what do you think of as like the canonical, um, best fits?
[00:46:40] Javi Fernandez Rico: Yeah, I think, um, right now the most use-- the, the biggest use case that we've seen for decision models are guardrails. That's probably the number one. Uh, detecting prompt injection, jailbreak, uh, harmful content, um, or just, like, personal data identification, stuff like that. That is very, uh... that's put in place for, uh, for decision models to, to, to react very quickly and identify those, and actually used by LLMs in harnesses to basically redact or remove things out of, of the identified, um, the identified data.
[00:47:22] There's also intent routing, so, uh, classifying the requests and sending it to the right agent instead of leveraging just the reasoning capabilities of the main agent. Because one of the difficulties, right, when, when you have a very capable model, that capable model is gonna try to solve the problem on its own, even if it has agents at its disposal.
[00:47:40] It's gonna be like, "Yeah, I'm gonna do it." And so if you have a decision model that is just routing the intent, uh, you can classify it and send it to the proper agent that has the proper access to tools and models instead of wasting cycles of the main agent just saying like, "Oh, I don't have access to this tool," and just, like, wasting cycles there, there.
[00:48:00] There's also tool call validation, so checking that the proposed agent actions are allowed. Um, so those, those probably are r- the biggest ones that I've seen. Um, triage and escalation is another one that it's very popular, uh, deciding whether or not this requires a human in the loop, uh, because there's, like, fraud
[00:48:22] Josh Rubin: Yeah.
[00:48:24] Javi Fernandez Rico: service, right?
[00:48:25] Like, "Hey, bring the representative. Don't, don't let the LLM continue." Uh, very popular as well.
[00:48:31] Josh Rubin: Very nice. Um, what was I gonna say about that? us, you know, we do a lot of, you know, I think our, our kind of bread and butter is observability, right? And, uh, being able to do... know, part of observability is ingesting traces and spans and telemetry from agents, right? Um, and the other piece of it is using models to score on various dimensions, you know, how well a conversation went or, you know, was there a hallucination in this turn, or, um, did the model properly, um, interpret the result of a tool call, or did the model make up the result of a tool call?
[00:49:08] And, you know, you want to be able to look at those things, you know, without skipping, you know... One, one thing that, you know, you could use a bigger, slower model or a more expensive model and sample traffic, but one of the things that we try to drive home is that, um, know, there are infrequent but sort of dangerous, uh, things that you wanna catch and, and sampling can be risky.
[00:49:33] Javi Fernandez Rico: Mm-hmm.
[00:49:33] Josh Rubin: wanna do this at scale, you need, you need these evaluators to run, uh, very efficiently. And that gives you like, you know, both the statistical overview of the performance of your application, but also like, um, allows you to identify the rare kind of anecdotal things that have happened that could be, um, infrequent and high risk.
[00:49:54] So for me, that's a, that's a really, a really sweet spot for... E- even aside from like guardrails. I think guardrails are kind of a, a really clear, um, spot. Um,
[00:50:04] Javi Fernandez Rico: Yeah, I think a good rule of thumb, I think a good rule of... Sorry. I, I think a good rule of thumb is, like, anywhere that your agent needs, like, a yes or no or which one decision that requires, like, a fast kind of millisecond decision, that's where you stick up a decision model, right? That would be, like, the rule of thumb, uh, of where, where, where they fit
Audience Q&A: LoRA vs. Knowledge Distillation
[00:50:26] Josh Rubin: Totally. Um, okay, so let's... I, I've got a question here from, uh, from Syed Amar. Um, he says, uh, you know, for agentic use cases, in order to use an SLM effectively, I think we could probably also... SLM, maybe we can... It may be the same rules for, for decision models.
[00:50:43] Javi Fernandez Rico: Mm-hmm.
[00:50:44] Josh Rubin: You know, what techniques are preferred? Do you like fine-tuning LoRA and variance? Do you like knowledge distillation? assume we have limited data
[00:50:53] Javi Fernandez Rico: Yeah, that's a great question. Um, so it depends on the domain itself. Uh, as, as I mentioned before, you, you want the good eval. So if you have the... If you start with good evals, then you can measure, like, bigger models that, uh, that are out there, that are open source, and see which ones perform really well, and then perform distillation from those models.
[00:51:14] That's, that's, that's very easy, low-hanging fruit win for you, where you can generate those traces out of those big models that you know perform well with your evals, and then you can distill that, uh, that knowledge, um, to, to the, to the smaller model. The LoRA, it's preferred because of its computational advantages, where training LoRA is extremely fast, and, like, it doesn't require too much hardware to train.
[00:51:43] So if you're constrained on hardware, then you'll start with, uh, with LoRA. But you mentioned it was, like, data was limited, so you will still need to, to Josh's point, data is very important, so you will still need to generate some data, either with synthetic or find some, some examples that you can use for, for the LoRA adapter, um, to do.
[00:52:05] So I would say it, I hate to give a depends answer, but it really, it depends on what you're building and how well the, the big models perform. I tend to lean more towards, like, distillation if I don't have data, because the model might be good enough. If I have data, then start with LoRA because it's faster.
[00:52:24] You'll get faster results quickly.
[00:52:26] Josh Rubin: I think the other sort of unsung superpower of LoRA, uh, is that, um, from a serving perspective, from the infrastructure perspective, it's very efficient to swap LoRA adapters in and out of a, a single set of base weights. You might have a, know, just numbers from our own experience, right? For us, like, you know, you... It may take, you know, five minutes to spin up a pod in an auto-scaling scenario where you're trying to pull a full s- a full model up and stick it on a GPU and, you know. If you can field like a single base model that is, you know, basically replicated across your inference infrastructure and swap in LoRA adapters, that can be done, you know, essentially real time without giving latency up. Um, and so it allows you to, you know, juggle what are essentially fine-tuned variants, uh, very efficiently. Um, and again, if you want to do any of this at scale, those, those infrastructure and serving problems end up being
[00:53:26] Javi Fernandez Rico: Yeah,
[00:53:26] Josh Rubin: and
[00:53:27] Javi Fernandez Rico: how we serve, that's how we serve on our end. We train LoRAs, we swap them very quickly against the base, and that's how we serve traffic to some of the customers that have fine-tuned on, on our end, 'cause we fine-tune LoRA adapters. So we just swap them, and it's fractions of milliseconds to swap them, and it's, like, very efficient hardware-wise because you always have, like, the warm base, and then you're just changing some, like, LoRA adapters to, to generalize the, to the generalized base model.
[00:53:54] So pretty efficient.
Audience Q&A: Edge Cases and Model Escalation
[00:53:56] Josh Rubin: We have another, here's another question. How should teams handle edge cases a test specific model wasn't built to catch?
[00:54:03] Javi Fernandez Rico: Hmm. Yeah, uh, that's a great question. So edge cases, out of distribution cases are gonna happen all the time. The recommendation that we do is, um, because you don't know them during training time, you need to keep monitoring what the model is doing, identifying... We were just talking about those failure modes earlier today.
[00:54:25] You identify those e- failure modes, keep recording the traces, keep recording the actual outcomes and what was the problem with them. And then after you have a few example of those, rerun a training job. If you're doing LoRA, as we were just discussing, it's very quick, right, to create a new LoRA adapter with more specialization out of those examples that you can expand with synthetic data.
[00:54:47] And then, uh, you cover those edge case relatively quickly rather than waiting for a base model to just, uh, cover them, which might never happen. So yeah.
[00:54:58] Josh Rubin: I, you know, I, uh, so I said, uh, one thing we've attempted to do with some recent product development, so we have this thing that we call, um, uh, dynamic model escalation. So we try to... The, the thing that we were discovering was that there were edge cases where our tiny models, and these are smaller than decision models, um, you know, kind of bird scale encoders. Um, you know, if you gave them an input that was out of distribution, then you can't rely on its downstream prediction probability to be reliable,
[00:55:28] Javi Fernandez Rico: Yeah
[00:55:30] Josh Rubin: are conditioned on the training data. so, um, you know, we started doing things like compare, um, like upstream embeddings from the models to training distributions,
[00:55:42] Javi Fernandez Rico: Interesting
[00:55:43] Josh Rubin: as a, as a kind of a... to, to provide a side mechanism where we could identify early whether this was a good input for our fast model, or we needed to send it off to a, a bigger model that was slower. And what we found was like, you know, for some of our customers, if you escalated something, you know, like 2 to 5% of the traffic for the outlier cases, um, you could handle all the rest of that traffic very fast.
[00:56:07] But the overall combined classifier performance was, you know, much, much higher. So I-- that was, you know, that's something that we, we spent a, you know, half a year kind of working on, and we have our first flavors of that fielded. But I'd, I'd love to see... You know, I think that's, that's a nice way to handle those appropriately in the interim before you can run your retrain cycle and, and update your, you know, um-
[00:56:33] Javi Fernandez Rico: Yeah, for failure, zero failures, uh, it's always convenient to have a backup model that you can route the low confidence scores to that model and get an output out of it. It's, it's a great idea to, to actually do the embedding space to figure out like, okay, is this belongs to the distribution or not?
[00:56:51] And if it's not, then you get that quick signal to make the decision to move to the bigger model. So that's, uh
[00:56:57] Josh Rubin: we like to... You get that for free, right? That, yeah, that's, that's, that's just upstream of the, the fast classifier. So, uh, it's easy to, to make that decision. So that was, that's, that's been a fun thing to, you know, to, to start rolling out. I think we're really close to time. I don't know if you have any other sort of words that if, if I missed anything that, that you like to talk about.
[00:57:16] Uh, there's probably another minute or so if you wanna
Closing Thoughts
[00:57:20] Javi Fernandez Rico: Um, not really, uh, unless I can put a plugin on Fastino. Yeah, it's,
[00:57:27] Josh Rubin: Yeah, I'm, I, I'm a fan of Fastino also. We've been using GLiNER within our, um, sensitive information detection for, I don't know, a year and a half, and it's been, it's been very effective,
[00:57:37] So,
[00:57:37] Javi Fernandez Rico: like, if people are interested in fine-tuning or they just want to explore, like, decision models or smaller models, just check, check our, our Hugging Face, check our website. I think, uh, you'll, you'll, you'll, you'll have enough information there to figure out, like, if it's a good fit for what you need.
[00:57:53] So yeah, it was great. It was great to, to chat.
[00:57:57] Josh Rubin: Thanks a lot Javi.
[00:57:57] Javi Fernandez Rico: Yeah
[00:57:58] Josh Rubin: e- easy, easy, uh, hour-long conversation
[00:58:01] Javi Fernandez Rico: Yeah, I will... I could have stayed here for longer.
[00:58:03] Josh Rubin: Oh, me too, for sure, for sure.
[00:58:05] Javi Fernandez Rico: It was all right.
[00:58:07] Josh Rubin: Talk to you soon. Take care everybody. Thanks for dialing in.
[00:58:09] Javi Fernandez Rico: Thank you.
[00:58:10] Josh Rubin: Bye

