Adept

Excited to share that for the second year, Adept has been listed on the Forbes AI 50. Big thanks to our incredible team for all the hard work! Check out our open roles to join us! https://lnkd.in/gTwYQfpw https://lnkd.in/eT2MhX_f

Introducing Fuyu-Heavy, our newest multimodal model. Fuyu-Heavy is the world’s third-most-capable multimodal model, behind only GPT4-V and Gemini Ultra, which are 10-20 times larger. In particular, it outperforms Gemini Pro at both MMLU and MMMU. Training Fuyu-Heavy wasn’t easy - in addition to the standard hiccups with model scaling, we had to deal with the extra problems associated with training a new architecture on both text and image data. Here’s a blog post if you want all the details: https://lnkd.in/gHeUbqRn We’re working on further scaling up these models and building useful software agents around them - if that sounds exciting to you, please reach out at https://lnkd.in/eT2MhX_f

Today we’re opening access to Adept Experiments 🧪, a new way to explore the technology we are developing at Adept. Each experiment is a self-contained mini-tool or demo that showcases a part of our underlying tech. Read more here: https://lnkd.in/erg2jH5B The first Experiment we're rolling out is Workflows ⚡️ A foundational skill for an AI teammate is to 1) quickly learn a task from a user and 2) reliably run it. Workflows are designed around this philosophy. So... what can workflows do? Workflows can do repetitive tasks, turn unstructured data into structured data, trivially hop between tools, or even teach a novice user to navigate the tool like an expert. The example video is a workflow that can turn unstructured data into structured data— Adept opens attached invoices, extracts information, and moves it into accounts payable software. Workflows is powered by ACT-2, a multimodal model fine-tuned from the Fuyu family and optimized for UI understanding, knowledge worker data comprehension, and action taking. Because Workflows is an Experiment, it often requires some careful prompting to make it do what you want. Some sites work better than others. Stay tuned for more Experiments and research updates as we continue to iterate. Get on the waitlist at https://lnkd.in/e7kThAAH. We're onboarding folks on a rolling basis. Excited to see what you build! 🚀

We’re open-sourcing a multimodal model: Fuyu-8B! Building useful AI agents requires fast foundation models that can see the visual world. Fuyu-8B performs well at standard image understanding benchmarks, but it also can do a bunch of new stuff. We think MM models are especially useful for handling unstructured knowledge worker data, so we’ve given Fuyu-8B capabilities in: - Understanding diagrams, charts, and graphs - Doing OCR on screens - Outputting bounding boxes for the locations of objects on screens - Answering UI-based questions It actually has an extremely simple architecture. Fuyu-8B doesn’t have an image encoder. This allows easy interleaving of text and images and handling arbitrary image resolutions! And it’s super fast for copilot use cases where latency really matters. Weights are here https://lnkd.in/ehBWiMRF, and you can also try a demo https://lnkd.in/eN-cXHwE. We can’t wait to see what you build on top of it! Read more on our blog: https://lnkd.in/eSCJhDHe