Fast AI, European rules: how Infercom brings inference home

[This article is part of a content series developed in collaboration with FEDIL, showcasing how artificial intelligence is contributing to the digital transformation of Luxembourg’s economy.]

Infercom is an EU sovereign AI inference provider, that was created in Luxembourg in August  2025. It runs open-weight models  (an artificial intelligence model whose numerical parameters, known as “weights”, learned during training are made publicly available for download) on its own purpose-built AI chips, in a data center in Munich, for companies that build products and processes on AI.

The idea behind Infercom genesis came with a frustration that many European organisations share. "European companies wanted to use AI seriously, but almost every powerful option meant sending their data to a US hyperscaler", explains Altug Eker, Managing Director of Infercom.

Good ideas that stalled

For banks, insurers or public institutions, the location and legal control of data can shape whether an AI project is feasible. With the General Data Protection Regulation (GDPR), the Schrems II ruling (a Court of Justice of the European Union decision that invalidated the “Privacy Shield”, which had facilitated the transfer of personal data from the EU to the United States), the Digital Operational Resilience Act (DORA) and now the EU AI Act, companies are subject to an increasing number of constraints and restrictions, which do not prevent some central questions from being asked: "Who can access my data? Under which law do I have to operate?"

Data governance is not the only hurdle. Speed in another major barrier to many initiatives. Some teams had promising ideas but struggled to turn them into practical services. Slow responses undermined the user experience, costs became harder to manage as demand grew, and infrastructure located outside Europe raised further concerns.

Mr Eker heard the same request from customer after customer: "We want AI, but we want it here, under our rules, and we want it fast." That’s why Infercom was created to answer it.

Inference, explained and then accelerated

Infercom is a European sovereign AI inference platform. The inference is the stage when a trained model is put to work. Training teaches a model; inference is what happens when someone asks it a question, uses it to review documents or relies on it to complete a task.

"Every chatbot reply, every summarised contract, every AI agent running a task is inference", Mr Eker explains. "It is the part companies use every single day, and it is where speed, cost and data control really matter."

For a CFO, AI stops being an unpredictable cloud bill and becomes a cost they can plan and manage. Altug Eker, CEO of Infercom

The Infercom platform runs open-weight models, including MiniMax, Gemma and GLM, on SambaNova Reconfigurable Dataflow Unit (RDU) hardware hosted in Munich. The chips are designed for AI workloads, and the platform runs the models at full precision rather than applying quantisation to reduce computing demands.

Organisations can connect through an application programming interface (API). Those requiring a dedicated environment can use specific racks instead. In either case, the customer data remains in the European Union, is processed by a Luxembourg company and falls under European law.

A bill that finance teams can plan

Infercom’s pricing is based on tokens, the units of text that a model processes or produces. Rather than renting graphics processing unit (GPU) capacity that may sit unused, customers pay for the activity they really carry out.

The platform also provides tools to monitor usage and set spending limits. For tasks that do not need an immediate response, such as batch processing or document analysis overnight, off-peak pricing offers another way to manage expenditure. “For a CFO, AI stops being an unpredictable cloud bill and becomes a cost they can plan and manage”, Mr Eker notes.

What are the benefits?

The platform is designed to bring together responsiveness, European data control and a more predictable cost model.

• Faster responses for live interactions - Infercom claims up to 10x faster inference than typical GPU-based setups, with the first token generated in under 10 milliseconds. “In practice, the model starts answering almost before you finish asking”, Mr Eker points out. This responsiveness is particularly relevant to voice-based applications. A long pause can disrupt a conversation, while a quick reply helps an AI assistant keep pace with the person speaking. “A voice assistant built on Infercom does not pause, hesitate or talk over you. The conversation flows seamlessly, just like talking to a real human, without awkward silences or interruptions.”
  For customer service, sales calls and other real-time AI agents, Mr Eker sees the difference as “between a tool people tolerate and an experience people actually enjoy”.

“We want to be the European alternative to US hyperscalers for inference, not a copy. Altug Eker, CEO of Infercom

• European data handling and certification - Data that Infercom treats is hosted and processed in Europe. The company is ISO 27001 and CSA STAR for AI certified. For organisations operating in regulated sectors, these measures can help address questions about data handling and security. Mr Eker says they “make the compliance conversation much shorter for regulated industries”.
• Costs that can be managed as use grows - Pay-as-you-use pricing, spending controls and off-peak options are intended to give organisations more visibility as their workloads expand. The platform is also designed to give customers a choice of models and infrastructure rather than tying them to one provider. “We want to be the European alternative to US hyperscalers for inference, not a copy.”

What are the next steps?

Infercom’s next development phase centres on making its infrastructure more flexible and better suited to different AI tasks. “We are just getting started”, says Mr Eker. The company plans to move beyond relying on one kind of chip or deployment arrangement. SambaNova RDUs will remain part of the platform, alongside GPUs and emerging European AI accelerators.

Infercom also intends to link its cloud infrastructure with customers’ on-premises systems, for cases where data needs to remain within an organisation’s own environment. This would make the platform less dependent on any single hardware supplier and allow customers to combine cloud and on-premises resources according to their needs.

Infercom also plans to divide inference into separate stages and allocate each stage to the hardware best suited to it. Processing a lengthy prompt and generating a response are different workloads, but providers commonly handle them on the same machine. Separating the stages could allow the platform to use computing resources more efficiently. Infercom expects this approach to support quicker responses and reduce the cost of processing each token.

A system that assigns different tasks to different chips could also make it easier to introduce new European processors. Rather than requiring each chip to handle every part of inference, the architecture could use it for the tasks it performs most effectively. The company plans to invest in and work with European AI accelerator chipsets. “Real digital sovereignty needs European hardware, not only European data centres”, Mr Eker stresses.

A multi-location and multi-node infrastructure is also under development. Operating across several European sites could improve resilience and reduce response times for customers in different countries. It could also provide options for keeping data within particular national borders where required. Altug Eker summarises the goal as “fast, trusted, European AI infrastructure that any company here can build on with confidence.”