Connect with us


Rabbit is building an AI model that understands how software works



What if you could interact with any piece of software using natural language? Imagine typing in a prompt and having AI translate the instructions into machine-comprehendable commands, executing tasks on a PC or phone to accomplish the goal that you just described?

That’s the idea behind Rabbit, a rebranding of Cyber Manufacture Co., which is building a custom, AI-powered UI layer designed to sit between a user and any operating system.

Founded by Jesse Lyu, who holds a bachelor’s degree in mathematics from the University of Liverpool, and Alexander Liao, previously a researcher at Carnegie Mellon, Rabbit is creating a platform, Rabbit OS, underpinned by an AI model that can — so Lyu and Liao claim — see and act on desktop and mobile interfaces the same ways that humans can.

“The advancements in generative AI have ignited a wide range of initiatives within the technology industry to define and establish the next level of human-machine interaction,” Lyu told TechCrunch in an email interview. “Our perspective is that the ultimate determinant of success lies in delivering an exceptional end-user experience. Drawing upon our past endeavors and experiences, we’ve realized that revolutionizing the user experience necessitates a bespoke and dedicated platform and device. This fundamental principle underpins the current product and technical stack chosen by Rabbit.”

Rabbit — which has $20 million in funding contributed by Khosla Ventures, Synergis Capital and Kakao Investment, which a source familiar with the matter says values the startup at between $100 million and $150 million — isn’t the first to attempt a layering natural language interface on top of existing software.

Google’s AI research lab, DeepMind, has explored several approaches for teaching AI to control computers, for example having an AI observe keyboard and mouse commands from people completing “instruction-following” tasks such as booking a flight. Researchers at Shanghai Jiao Tong University recently open sourced a web-navigating AI agent that they claim can figure out how to do things like use a search engine and order items online. Elsewhere, there’s apps like the viral Auto-GPT, which tap AI startup OpenAI’s text-generating models to act “autonomously,” interacting with apps, software and services both online and local, like web browsers and word processors.

But if Rabbit has a direct rival, it’s probably Adept, a startup training a model, called ACT-1, that can understand and execute commands such as “generate a monthly compliance report” or “draw stairs between these two points in this blueprint” using existing software like Airtable, Photoshop, Tableau and Twilio. Co-founded by former DeepMind, OpenAI and Google engineers and researchers, Adept has raised hundreds of millions of dollars from strategic investors including Microsoft, Nvidia, Atlassian and Workday at a valuation of around $1 billion.

So how does Rabbit hope to compete in the increasingly crowded field? By taking a different technical tack, Lyu says.

While it might sound like what Rabbit’s creating is akin to robotic process automation (RPA), or software robots that leverage a combination of automation, computer vision and machine learning to automate repetitive tasks like filing out forms and responding to emails, Lyu insists that it’s more sophisticated. Rabbit’s core interaction model can “comprehend complex user intentions” and “operating user interfaces,” he says, to ultimately (and maybe a little hyperbolically) “understand human intentions on computers.”

“The model can already interact with high-frequency, major consumer applications — including Uber, Doordash, Expedia, Spotify, Yelp, OpenTable and Amazon — across Android and the web,” Lyu said. “We seek to extend this support to all platforms (e.g. Windows, Linux, MacOS, etc.) and niche consumer apps next year.”

Rabbit’s model can do things like book a flight or make a reservation. And it can edit images in Photoshop, using the appropriate built-in tools.

Or rather, it will be able to someday. I tried a demo on Rabbit’s website and the model’s a bit limited in functionality at the moment — and it seems to get confused by this fact. I prompted the model to edit a photo and it instructed me to specify which one — an impossibility given that the demo UI lacks an upload button or even a field to paste in an image URL.

The Rabbit model can indeed, though, answer questions that require canvassing the worldwide web, a la ChatGPT with web access. I asked it for the cheapest flights available from New York to San Francisco on October 5, and — after about 20 seconds — it gave me an answer that appeared to be factually accurate, or at least plausible. And the model correctly listed at least a few TechCrunch podcasts (e.g. “Chain Reaction”) when asked to do so, beating an early version of Bing Chat in that regard.

Rabbit’s model was less inclined to respond to more problematic prompts such as instructions for making a dirty bomb and one questioning the validity of the Holocaust. Clearly, the team’s learned from some of the mistakes of large language models past (see: the early Bing Chat’s tendency to go off the rails) — at least judging by my very brief testing.


The demo model on Rabbit’s site, which is a bit limited in functionality.

“By leveraging [our model], the Rabbit platform empowers any user, regardless of their professional skills, to teach the system how to achieve specific goals on applications,” Lyu explains. “[The model] continuously learns and imitates from aggregated demonstrations and available data on the internet, creating a ‘conceptual blueprint’ for the underlying services of any application.”

Rabbit’s model is robust to a degree to “perturbations,” Lyu added, like interfaces that aren’t presented in a consistent way or that change over time. It simply has to “observe,” via a screen-recording app, a person using a software interface at least once.

Now, it’s not clear just how robust the Rabbit model is. In fact, the Rabbit team doesn’t know itself — at least not precisely. And that’s not terribly surprising, considering the countless edge cases that can crop up in navigating a desktop, smartphone or web UI. That’s why, in addition to building the model, the company’s architecting a framework to test, observe and refine the model as well as infrastructure to validate and run future versions of the model in the cloud.

Rabbit also plans to release dedicated hardware to host its platform. I question the wisdom of that strategy, given how difficult scaling hardware manufacturing tends to be, the consumer hostileness of vendor lock-in and the fact that the device might have to eventually compete against whatever OpenAI’s planning. But Lyu — who curiously wouldn’t tell me exactly what the hardware will do or why it’s necessary — admits that the roadmap’s a bit in flux at the moment.

“We are building a new, very affordable, and dedicated form factor for a mobile device to run our platform for natural language interactions,” Lyu said. “It’ll be the first device to access our platform … We believe that a unique form factor allows us to design new interaction patterns that are more intuitive and delightful, offering us the freedom to run our software and models that the existing platforms are unable to or don’t allow.”

Hardware isn’t Rabbit’s only scaling challenge, should it decide to pursue its proposed hardware strategy. A model like the one Rabbit’s building presumably needs a lot of examples of successfully completed tasks in apps. And collecting that sort of data can be a laborious — not to mention costly — process.

For example, in one of the DeepMind studies, the researchers wrote that, in order to collect training data for their system, they had to pay 77 people to complete over 2.4 million demonstrations of computer tasks. Extrapolate that out, and the sheer magnitude of the problem comes into sharp relief.

Now, $20 million can go a long way — especially since Rabbit’s a small team (9 people) currently working out of Lyu’s house. (He estimates the burn rate at around $250,000.) I wonder, though, whether Rabbit will be able to keep up with the more established players in the space — and how it’ll combat new challengers like Microsoft’s Copilot for Windows and OpenAI’s efforts to foster a plugin ecosystem for ChatGPT.

Rabbit is nothing if not ambitious, though — and confident it can make business-sustaining money through licensing its platform, continuing to refine its model and selling custom devices. Time will tell.

“We haven’t released a product yet, but our early demos have attracted tens and thousands of users,” Lyu said. “The eventual mature form of models that the Rabbit team will be developing will work with data that they have yet to collect and will be evaluated on benchmarks that they have yet to design. This is why the Rabbit team is not building the model alone, but the full stack of necessary apparatus in the operating system to support it … The Rabbit team believes that the best way to realize the value of cutting-edge research is by focusing on the end users and deploying hardened and safeguarded systems into production quickly.

Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *


India’s Private Rocket ‘Agnibaan’ Set For Launch By Agnikul Cosmos On May 28




Another private Indian rocket is launching on May 28. Agnikul Cosmos will attempt to launch the ‘Agnibaan’ for the second time tomorrow at an expected time of between 5:30 to 7:30 am IST, India Today reported.

The Chennai-based space startup has also issued a Notice to Airmen (Notam) information to aware the authorities about the potential launch to avoid hazards.

According to Agnikul, the Notam will be active till June 5 for the launch which will take place from the company’s spaceport ALP-01 at Satish Dhawan Space Center in Sriharikota. This spaceport, which is India’s first private one, was inaugurated by ISRO Chairman S Somanath on November 25, 2023.

This would mark the second mission with a commercial launch vehicle. India’s first private rocket Vikram-S flew on November 18, 2022 during Skyroot Aerospace’s Prarambh mission.

ALSO SEE: India Gets Its First Private Launch Pad, And It Will Give Space Sector A Major Boost; Here’s How

What is Agnikul’s mission about?

Agnikul’s mission named SubOrbital Technological Demonstrator or SOrTeD is meant to test the single-stage Agnibaan rocket.

According to Agnikul’s website, the rocket to be used for SOrTeD will be powered by the Agnilet semi-cryogenic engine that uses kerosene and liquid oxygen. This single piece engine is entirely 3D printed and would be the first such power source used in a private rocket launch.

“Unlike traditional sounding rockets that launch from guide rails, Agnibaan SOrTeD will lift off vertically and follow a predetermined trajectory while performing a precisely orchestrated set of manoeuvres during flight,” Agnikul said in its mission profile.

ALSO SEE: Indian Tech Startup To Create World’s First 3D-Printed Rocket Engine; All You Need To Know

Agnikul is developing Agnibaan as a two-stage rocket which can be customised according to the customer’s needs. This two-stage rocket which measures 18 meters tall has a maximum payload capacity of 100 kg to a 700 km orbit and can be customised to add a ‘baby’ stage.

The mission was initially planned for launch on April 7 but was called off about two minutes before the lift off due to a communication issue between onboard hardware.

(Image: Agnikul Cosmos)

Source link

Continue Reading


Five Asteroids In 3 Days! A Barrage Of Space Rocks Are Heading Toward Earth This Week




Five asteroids of varying sizes are set for a close encounter with Earth in the next two days. NASA’s Center For Near-Earth Object Studies (CNEOS) has revealed that the smallest of them measures between roughly 4.5 to 10 meters in diameter whereas the biggest of them is about 32 to 73 meters.

The data has revealed that the smallest – 2008 LD – will be approximately 29.5 lakh km from Earth at the time of the fly by. It is travelling at a speed more than 16,000 km per hour.

Orbit of asteroid 2024 JV17. Image: NASA

The biggest of the five – the 2024 JV17 – will be approximately 66 lakh km away at the time of the closest approach while travelling at over 30,000 km per hour. Both these space rocks will fly past Earth on May 28.

ALSO SEE: Like Dinosaurs, Humans Will Become Extinct If A Single Asteroid Collides; ‘Asteroid Rush’ Trailer Proves Just That

The other asteroids – the 2021 LV (between 7-15 meters wide) and 2024 JG (between 22-50 meters) will get close to our planet on May 29.

Orbit of asteroid JO16. Image: NASA

There will be a close encounter today as well when the asteroid 2024 JO16 flies past Earth. According to the CNEOS data, it will be about 30 lakh km from our planet and will be travelling at more than 30,000 km per hour.

As the data suggests, there is no need to worry since the asteroids will fly from a safe distance from our planet. Besides, their size except for a few relatively big ones is also not a cause for concern.

ALSO SEE: Lack Of Earth’s Resources Can Be Fulfilled By Asteroids – And A US Firm Wants To Mine Them

(Image: Unsplash)

Source link

Continue Reading


Paytm warns of job cuts as losses swell after RBI clampdown




Indian digital payments platform Paytm warned of job cuts on Wednesday after reporting that its net loss widened in the fourth quarter as it grapples with a recent regulatory clampdown.

One97 Communications, Paytm’s parent, said it expects to cut employee expenses and pare down its annual staff costs by $48 million to $60 million.

The company, once the most valuable Indian startup, reported a net loss of $66.1 million in the fourth quarter ended March 2024, compared to a loss of $20.11 million a year earlier. Revenue declined about 3% to $272.4 million from $280.4 million in the same period.

India’s central bank in February banned the company’s banking partner and sister company, Paytm Payments Bank, from conducting banking activity from March. That brought a sudden halt to Paytm’s slew of banking services, and the company was forced to ink new partnerships with other banks to keep many of those services running.

Paytm said it also took an impairment charge of $27.2 million related to its investment in Paytm Payments Bank in the quarter. In the quarter ending June this year, Paytm projected its revenue to be in the range of $180 million to $192 million.

In the full year ended March, Paytm’s revenue increased 25% to $1.19 billion from a year earlier, though higher payment processing charges, marketing costs, employee benefits charges and software cloud expenses weighed on its bottom line. As a result, net loss widened to $170 million from a loss of $213 million a year earlier.

Paytm’s results include “enough data points to suggest that the business is past the bottom in terms of payment volumes and user/merchant traction,” Bernstein analysts said in a note to clients. “Though from a financial metrics perspective, 1QFY25 is likely to be the bottom, as it would reflect the full impact of the lower steady state (vs. 2 months impact in 4QFY24).”

The analysts, however, cautioned that Paytm’s payment GMV has dropped by about 20% and the company’s expectations for its payment processing margin has also declined, which together “translates to a near 50% blow to the payment margins.” They estimated, however, that Paytm’s merchant lending volumes picked up in March and April — a clear sign of revival.

Paytm had about $1.03 billion in the bank as of March 31. The company’s shares were down about 1% on Wednesday afternoon to ₹349.20, giving it a market cap of $2.64 billion. Paytm went public in 2021 at a valuation of $20 billion.

“I am happy to share that we have successfully transitioned our core payment business from PPBL to other partner banks. This move de-risks our business model and also opens up new opportunities for long-term monetization, given our platform’s strength around customer and merchant engagement,” said Paytm’s founder and CEO, Vijay Shekhar Sharma, in the company’s annual shareholder letter.

“It has been possible in such a short period of time with extensive support from the Regulator, NPCI, Bank partners and our committed team mates. The unwavering commitment of our government and regulator to support innovation and financial inclusion, keeps us true to our mission and committed to our long-term sustainable growth opportunity,” he added.

Source link

Continue Reading


Copyright © 2023 Dailycrunch. & Managed by Shade Marketing & PR Agency