Guest author: Ryohei Fujimaki, founder and CEO, dotData
AI is a double-edged sword. On the one hand, the technology can unlock billions of dollars a year for companies, as a Google report about opportunities in the UK reveals. But on the other hand, misuse, security issues, privacy breaches, and bias present serious risks.
The solution for business leaders embracing AI’s disruptive power is to isolate their AI systems on the cloud, edge, or on-premises environments. AI isolation is critical to guarantee infrastructure integrity, customize models to increase business efficiencies, and strengthen cybersecurity and privacy.
What follows is a practical technical and business guide for companies to achieve AI isolation today.
A technical blueprint for AI isolation
Several virtual machines can run completely isolated operating systems on the same physical data center; similarly, AI systems can be isolated from any infrastructure. This can be accomplished by using various virtualization methods and software-level solutions.
Step 1: Choose your VM provider and AI isolation technology
Most AI systems run on virtual machines (VMs). Therefore, companies must understand how isolation layers are constructed, evaluate the VM market and options, and dive into additional technologies used to build unique architectures.
Whether deploying the AI on-premise, on the edge, or in the cloud, VMs need to be created to isolate AI systems. VMs are built and managed using hypervisor software. There are several excellent VM providers, including VMware, Citrix, Red Hat, Microsoft, Oracle, and others. They all offer cost-efficient solutions to develop, deploy, and manage VMs on-premises or in the cloud.
When choosing a VM provider, companies must understand the demands of their AI systems because, when the VM is created, it is assigned resources such as storage space, memory, and other processing components. Additionally, who will use the AI must also be defined from the outset, as network configurations are determined when initially creating VMs.
AI can be isolated in different ways within VMs via sandboxes or containers.
Sandboxes
Sandboxes are isolated environments that run in a VM or a container. These can be a good choice to isolate AI technology for internal or experimental use. The downside is that sandboxes can be technically challenging to set up, and maximizing performance can be complex.
Containers
Containers like sandboxes typically limit the resources that an application can access, such as CPU, memory, and storage, increasing isolation security. However, containers are a better virtualization technology when isolating AI systems. Containers are lightweight, improve app performance, and allow applications to run on the same host machine.
Step 2: Design the isolated environment and deploy the VM
When designing the isolated environment, companies must choose whether to deploy AI on the edge, the cloud, or on-premises. Their choice will depend on their business objectives. A business developing AI for all its employees to use might prefer using the cloud to facilitate access. In contrast, another organization might want to use AI to engage with customers and, therefore, turn to edge-cloud or edge-on-premises options to provide more instant communications.
Once the infrastructure is chosen, the VM must be deployed within it. The VM resources are assigned, and its basic configuration is set up.
Step 3: Deploy isolation security components
Setting the security foundation is next. Different technologies to protect systems from unauthorized access and manipulation must be considered. Companies that host their own server or data centers, such as cloud providers, need to secure the physical location where their infrastructure is running. This means making sure only authorized users can physically access the site.
Access control and encryption
Network security and access control policies are critical for cloud and edge architectures. Companies must deploy access control solutions that restrict who can access the AI, such as role-based access control (RBAC) and network segmentation. Encrypting the models and data transfer also adds a layer of security to the system. Encryption can be symmetric, asymmetric, or hardware-based security encryption.
Security solutions
Endpoint security solutions are critical to visualize which user and device are connected to the system. And network security, next-generation firewalls, intrusion detection systems (IDS), and intrusion prevention systems (IPS) add layers that strengthen the system.
Companies can also deploy other solutions, such as threat intelligence — software that can help identify, flag, and shut down attacks — to ensure that VMs, where the AI is contained, are free of threats.
Back-up and recovery
Finally, setting backup and recovery technologies is crucial when designing isolation environments. Backups should be scheduled regularly to assist in the rapid recovery of systems in the face of incidents.
Step 4: Design, deploy and test the AI in the isolated environment
Sourcing and building the AI
Depending on business objectives and resources, a company might design its own AI or outsource it. Costs, scalability, security, and customization must be evaluated against options.
How AI was built is also important. For example, open-source deep learning frameworks, such as TensorFlow, PyTorch, or Keras, allow high levels of customization. Other AI solutions, such as those offered by leading cloud vendors, leverage the latest cybersecurity and automation. Finally, businesses can also turn to AI or ML as a Service, such as H2O.ai, DataRobot, Databricks, Hyperscience and others. These solutions can be very efficient as they are developed specifically for a business and not necessarily expensive.
Deployment and testing
In this stage, the AI is installed and deployed in an isolated environment. The application must be configured, and performance must be checked. Companies should follow up with a series of tests to make sure the AI is operational, the resources added are sufficient, and security components are responding as planned.
The final test of this step should be a small real-pilot test for users. During this test, a select number of users for whom the AI was designed must be engaged. Tests are monitored and later analyzed along with user feedback to improve the technology.
Step 5: Communicate and move the AI to production
An integral communication plan must be operating before an AI system is deployed. Users and third parties must be informed on how the AI system works and what it was designed for, with the appropriate onboarding and support guides made available.
A company also needs to establish a multi-channel support system and provide ways for users to give feedback. Support can be offered through chat, email, and phone.
The best practice for moving an AI system to production is to do it gradually. A company can take a department-by-department or project-by-project approach. The idea is simple. While an AI business strategy needs to be “big”, a company should take small wins. This will mitigate the risks that come with large-scale software and application releases and help teams build up confidence, trust, and momentum.
Step 6: Monitor and adjust
Once the AI system is deployed, a company must assign monitoring and adjustment responsibilities to its IT teams and related departments. Security, performance, scalability, and communications must be monitored, and updates and adjustments must be made when required.
Common misconceptions about building an isolated but connected AI
Just because an AI app is isolated, this does not mean that it is disconnected. Businesses can still reap the security and customization benefits of isolation while offering a large number of users access to AI. They can also connect the AI to external databases and even the internet, if needed, without compromising integrity.
Connected isolation can be achieved through secure APIs, proxy servers, security solutions, and virtual private networks (VPNs).
Another common misconception when setting up AI systems is performance. Distributed architecture models divide a system into multiple parts, each hosted on a different server. This allows the model to scale and handle a large number of requests simultaneously, meaning companies can create an efficient and no-delay AI system by avoiding processing requests individually from each user.
Within distributed models, a “beehive-mind” style-structure can be used to connect each part of a distributed system, allowing the different independent components of the AI app to share information and collaborate on tasks. This is the structure and shape that generative AI apps like Google’s Bard, ChatGPT, Bing, and others use.
The scalability of distributed models is extremely powerful and capable of handling millions of requests per minute—but only if the required infrastructure is in place. These models are a good fit for businesses that want to deploy AI systems at large scale for global customers. Additionally, distributed architecture models are reliable because if one server fails, another can handle the request.
Final thoughts for business leaders
The only way enterprises can safely use AI technology for business-critical objectives is to implement isolated AI systems. These will play an increasingly more critical role in the protection of your business partners and customer data, especially as new laws like the EU AI Act move forward.
By taking a more hands-on approach to AI, companies can tailor the systems to meet specific needs, improve accuracy and enhance their data-driven efforts. And by taking ownership of AI technology correctly and responsibly, companies can improve security, privacy, and performance, allowing them to achieve their goals and generate new value.
Ryohei Fujimaki is the founder and CEO of dotData, a company that makes it simple for organizations of any size to leverage the power of their data through fast, unique, and easy-to-use tools.
