OpenStack is a cloud OS that is used to control the large pools of computing, storage, and networking resources within a data center. Open Stack is open-source and free software. This is basically used for cloud computing and deployed as an IaaS.
In our OpenStack tutorial, we are going to discuss the following topics -
What is Cloud Computing
Cloud computing means storing and manipulating programs and data over the internet instead of the hard disk of your computer. This means that, instead of storing the files on a hard drive or local storage unit, we can save the files on a remote database via cloud-based storage.
In other words, we can say cloud computing means the delivery of computing services, such as servers, software, networking, analytics, intelligence, and databases over the internet or the cloud. It offers flexible resources, faster innovation, and economics of scale.
Types of Cloud Computing
Three major forms of cloud computing exist. These are:
Infrastructure as a Service (IaaS): It is the basic cloud service that offers networking services, load balancers, virtual machines, and firewalls services. It includes a method of providing everything from OS to servers and storage via IP-based networking as part of an on-demand service. Some common examples of IaaS are IBM cloud, AWS, and Microsoft Azure.
Platform as a Service (PaaS): A cloud computing service that offers an on-demand platform for software application development, management, testing, and distribution. If you use PaaS services, then you don't have to worry about setting up or maintaining the underlying server, network, storage, and database infrastructure required for the development. Example of PaaS is Google App Engine, Salesforce.com, etc.
Software as a Service (SaaS): It is a distribution model. Through this service, computer applications (web services) are distributed over the Internet. Users may use a computer or mobile device that has internet connectivity to access SaaS services. The most common example of a SaaS is Microsoft Office 365, which provides productivity and email services.
You can visit our Cloud Computing Tutorial to learn about cloud computing.
Introduction to OpenStack
OpenStack is a cloud OS that is used to control the large pools of computing, storage, and networking resources within a data center. OpenStack is an open-source and free software platform. This is essentially used and implemented as an IaaS for cloud computing.
We can call the OpenStack a software platform that uses pooled virtual resources to create and manage private and public cloud. OpenStack offers many cloud-related services (such as networking, storage, image services, identity, etc.) by default. This can be handled by users through a web-based dashboard, a RESTful API, or command-line tools. OpenStack manages a lot of virtual machines; this permits the usage of physical resources to be reduced.
Basic Principles of OpenStack
Open Source: Under the Apache 2.0 license, OpenStack is coded and published. Apache allows the community to use it for free.
Open Design: For the forthcoming update, the development group holds a Design Summit every 6 months.
Open Development: The developers maintain a source code repository that is freely accessible through projects like the Ubuntu Linux distribution via entig100s.
Open Community: OpenStack allows open and transparent documentation for the community.
Components of OpenStack
Major components of OpenStack are given below:
Compute (Nova): Compute is a controller that is used to manage resources in virtualized environments. It handles several virtual machines and other instances that perform computing tasks.
Object Storage (Swift): To store and retrieve arbitrary data in the cloud, object storage is used. In Swift, it is possible to store the files, objects, backups, images, videos, virtual machines, and other unstructured data. Developers may use a special identifier for referring the file and objects in place of the path, which directly points to a file and allows the OpenStack to manage where to store the files.
Block Storage (Cinder): This works in the traditional way of attaching and detaching an external hard drive to the OS for its local use. Cinder manages to add, remove, create new disk space in the server. This component provides the virtual storage for the virtual machines in the system.
Networking (Neutron): This component is used for networking in OpenStack. Neutron manages all the network-related queries, such as IP address management, routers, subnets, firewalls, VPNs, etc. It confirms that all the other components are well connected with the OpenStack.
Dashboard (Horizon): This is the first component that the user sees in the OpenStack. Horizon is the web UI (user interface) component used to access the other back-end services. Through individual API (Application programming interface), developers can access the OpenStack's components, but through the dashboard, system administrators can look at what is going on in the cloud and manage it as per their need.
Identity Service (Keystone): It is the central repository of all the users and their permissions for the OpenStack services they use. This component is used to manage identity services like authorization, authentication, AWS Styles (Amazon Web Services) logins, token-based systems, and checking the other credentials (username & password).
Image Service (Glance): The glance component is used to provide the image services to OpenStack. Here, image service means the images or virtual copies of hard disks. When we plan to deploy a new virtual machine instance, then glance allows us to use these images as templates. Glance allows virtual box (VDI), VMware (VMDK, OVF), Raw, Hyper-V (VHD) and KVM (qcow2) virtual images.
Telemetry (Ceilometer): It is used to meter the usage and report it to OpenStack's individual users. So basically, Telementry provides billing services to OpenStack's individual users.
Orchestration (Heat): It allows the developers to store the cloud application's necessities as a file so that all-important resources are available in handy. This component organizes many complex applications of the cloud through the templates, via both the local OpenStack REST API and Query API.
Shared File System (Manila): It offers storage of the file to a virtual machine. This component gives an infrastructure for managing and provisioning file shares.
Elastic Map-reduce (Sahara): The Sahara component offers a simple method to the users to preplanned Hadoop clusters by referring to the multiple options such as the Hadoop version, cluster topology and hardware details of nodes and some more.
Installation and Configuration of OpenStack
As we know, OpenStack is an open-source platform. OpenStack offers many options to install and deploy it through different software distributions. It provides various deployment models.
OpenStack also provides the pre-installed ISO images of the OpenStack platform. To download these pre-installed ISO images, click here.
Here, we will discuss the installation of the OpenStack using DevStack. Here, DevStack is the best option to install and test OpenStack on your PC, laptop, or server. DevStack installs all the components in one environment. It can be used to set up OpenStack on a virtual machine (VM) or a single server.
We cannot use the DevStack for production purposes because sometimes, after reboot, it is almost impossible to bring it up on the last state.
DevStack will install the following components:
The following are the hardware requirements for OpenStack:
DevStack can be installed on the following kind of operating systems:
Ubuntu operating system is the most tested and most reliable option for DevStack.
Installation of OpenStack
In order to install the DevStack in a system, first, you have to create a Linux VM on your computer (such as using VirtualBox or VMware) or remotely in the cloud (such as using AWS).
The VM must have at least 4GB of memory, and the proper internet connection is also important. Here, we are going to use one version of the ubuntu, i.e., 18.04.
Follow the following steps to install the OpenStack in your ubuntu virtual machine :
Step 1: Update Ubuntu System
Open the terminal and run the following command to ensure that the system is up to date :
Sample Output :
Reboot the system after running the above command. To reboot the system, run the following command :
Step 2: Create Stack User
It is important that the devstack must run as a regular user (non-root user) with the sudo enabled.
To keep this note in mind, let's create a new user with the name "stack" and assign the sudo permissions or privileges. To create a stack user, run the following command in your terminal:
Now, to assign the sudo privileges to the stack user, run the following command :
You can switch to the 'stack' user by running the following command:
Step 3: Install the Git
In Most of the ubuntu systems, git comes by default. But if git is missing on your system, then install it by running the following command:
Sample Output :
Step 4: Download OpenStack
Once you install the git, use the git command to download the DevStack from Github.
Step 5: Create a DevStack Configuration File
First of all, go to the devstack directory by running the following command :
Now, create a local.conf file in which you have to enter the four passwords and the host IP address :
Copy the following line of content in the file :
Press the ESC, then wq to save and then exit from the local.conf file.
Here, ADMIN_PASSWORD is the password that we will use to log into the OpenStack login page. The default username for an OpenStack is 'admin'.
And HOST_IP is the IP address of your system. To get your Server or VM IP, run the 'ifconfig' or 'ip addr' command.
Step 6 : Install OpenStack with DevStack
To install and run the openstack, execute the following command :
DevStack will install the following components:
The installation will take about 10-20 minutes, mostly depends on your internet speed.
At the very end of the installation, you will get the host's IP address, URL for managing it and the username and password to handle the administrative task.
Step 7: Accessing OpenStack on a browser
Copy the horizon URL given in the installation output and paste it into your browser :
To login to OpenStack with the default username - admin or demo and configured password - secret.
Once you login into the OpenStack, you will be redirected to the Dashboard of OpenStack. This dashboard screen is called the Openstack management web console.
Step 8: Create an Instance
On the main dashboard screen, you will see the instance's overview.
You can also create your own instance in the OpenStack. Instances are nothing but a virtual machine. To create a new virtual machine, click on the instances from the left side of the page.
And then click on Launch Instances. Fill in all the required fields. Once you fill all the required fields, an instance will create.
There are several components that form OpenStack's architecture.
Nova is one of the most common and important components of OpenStack. Compute is a controller that is used to handle virtualized environments' resources. It handles several virtual machines and other instances that perform computing tasks.
Nova is written in Python language. VMware, Xen, and KVM are the hypervisor technologies that could be used, and this choice is contingent on OpenStack's version.
This component is used for networking in OpenStack. Neutron manages all the network-related queries, such as IP address management, routers, subnets, firewalls, VPNs, etc. It confirms that all the other components are well connected with the OpenStack.
This component offers image services to OpenStack. Here, image service means the images or virtual copies of hard disks. When we plan to deploy a new virtual machine instance, glance allows us to use these images as templates.
Swift (Object Storage)
To store and retrieve arbitrary data in the cloud, object storage is used. In Swift, it is possible to store the files, objects, backups, images, videos, virtual machines, and other unstructured data.
Cinder (Block Storage)
This works in the traditional way of attaching and detaching an external hard drive to the OS for its local use. Cinder manages to add, remove, create new disk space in the server. This component provides the virtual storage for the VMs in the system.
This is the first component that the user sees in the OpenStack. Horizon is the web UI (user interface) component used to access the other back-end services.
It is the central repository of all the users and their permissions for the OpenStack services they use. This component is used to manage identity services like authorization, authentication, AWS Styles (Amazon Web Services) logins, token-based systems, and checking the other credentials (username & password).
How does OpenStack Work?
Basically, OpenStack is a series of commands which is called scripts. And these scripts are packed into packages, which are called projects that rely on tasks that create cloud environments. OpenStack relies on two other forms of software in order to construct certain environments:
So, we can say all three technologies, i.e., virtualization, base operating system, and OpenStack must work together.
Let's discuss how OpenStack works!
As we know, the Horizon is an interface for the appliance environment. Anything that the user wants to do should use the Horizon (Dashboard). The Dashboard is a simple graphical user interface with multiple modules, where each module performs specific tasks.
All the actions in OpenStack work by the service API call. So, if you are performing any task, it means you are calling a service API. Each API call is first validated by Keystone. So, you will have to login yourself as a registered user with your login username and password before you enter the OpenStack dashboard.
Once you successfully log in to the OpenStack dashboard, you will get many options to create new instances, volumes, Cinder, and configure the network.
Instances are nothing but a virtual machine or environment. To generate a new VM, use the 'instances' option from the OpenStack dashboard. In these instances, you can configure your cloud. Instances can be RedHat, OpenSUSE, Ubuntu, etc.
The formation of an instance is also an API call. You can configure network information in the instances. You can connect these instances to the cinder instance or volume to add more services.
After the successful creation of an instance, you can configure it, you can access it through CLI, and whatever data you want to add, you can do it. Even you can set up an instance to manage and store the snapshots for future reference or backup purposes.
OpenStack vs. AWS
OpenStack and AWS are both cloud computing platforms. Both serve different purposes. The AWS got introduced in 2006. And since 2006, the success of AWS API has been rising rapidly. Similarly, OpenStack had also grown since its beginning in 2010.
AWS (Amazon Web Services) is an on-demand cloud computing platform that can perform almost any kind of process. AWS offers a processing unit, storage, Big Data platform, ML platform, etc. Since AWS is an on-demand platform and charges very little, anyone can afford it according to their needs. It follows the concept of pay-as-you-go, but you just pay for the things you are using. Unlike OpenStack, AWS is not an open-source platform. Only as a user we can use it.
AWS is a public cloud and is mostly used by a small or medium level of companies who do cannot afford and don't want to spend on server infrastructure.
Highlights of AWS Services
Let's see some services of AWS:
Like AWS, OpenStack provides a cloud computing platform. Using OpenStack, we can perform all types of services.
But OpenStack is an open-source platform. Using OpenStack, we can create our own cloud and configure the things as per our needs.
We can deploy OpenStack in both private and public networks. Users can access the OpenStack via a web-based dashboard component or APIs.
Highlights of OpenStack
Difference between AWS and OpenStack
The difference between AWS and OpenStack usually depends on your company's specific requirements. Let's see the difference between OpenStack and AWS:
Benefits of OpenStack
There are a lot of benefits of OpenStack in the cloud computing platform. Let's see one by one :
1. Open Source
As we know, using the open-source environment, we can create a truly defined data center. OpenStack is the largest open-source platform. It offers the networking, computing, and storage subsystems in a single platform. Some vendors (such as RedHat) have developed and continue to support their own OpenStack distributions.
OpenStack source code is available at github. The two main advantages of the open-source OpenStack project is :
Scalability is the major key component of cloud computing. OpenStack offers better scalability for businesses. Through this feature, it allows enterprises to spin up and spin down servers on-demand.
One of the significant features of OpenStack is security, and this is the key reason why OpenStack is so popular in the cloud computing world.
Automation is one of the main keys selling points of OpenStack when compared to another option. The ease with which you can automate tasks makes OpenStack efficient. OpenStack comes with a lot of inbuilt tools that make cloud management much faster and easier. OpenStack provides its own API or Application Program Interface that helps other applications to have full control over the cloud. This function makes it easier to build your own apps that can communicate with OpenStack to perform tasks such as firing up VMs.
Since OpenStack's source code is freely accessible, experts from all over the world can improve the platform. If a new feature is being designed, it can be built easily and professionally by a development team. OpenStack is like Linux in that have many distributions with different features but share the same component.
Support from companies - For development, OpenStack gets support from IT founders, including Intel, IBM, AT&T, Cisco, Red Hat, Dell, Ubuntu, and so on. So, by leaps and bounds, it's changing, which is a massive benefit for you.
Support from the developers' community - Many developers are working on the enhancement of OpenStack. They are continuously working hard to make the OpenStack better.
5. Easy to Access and Manage
We can easily access and manage OpenStack, which is the biggest benefit for you. OpenStack is easy to access and manage because of the following features :
Command Line Tools - We can access the OpenStack using command-line tools.
Dashboard - OpenStack offers users and administrators to access and manage various aspects of OpenStack using GUI (graphical user interface) based dashboard component. It is available as a web UI.
APIs - There are a lot of APIs (Application Program Interface), which is used to manage OpenStack.
OpenStack provides many services required for several different tasks for your public, private, and hybrid cloud.
List of services - OpenStack offers a list of services or components such as the Nova, Cinder, Glance, Keystone, Neutron, Ceilometer, Sahara, Manila, Searchlight, Heat, Ironic, Swift, Trove, Horizon, etc.
Each component is used for different tasks. Such as Nova provides computing services, Neutron provides networking services, Horizon provides a dashboard interface, etc.
7. Strong Community
OpenStack has many experts, developers, and users who love to come together to work on the product of OpenStack and enhance the feature of OpenStack.
Public cloud systems like AWS (Amazon Web Services) are compatible with OpenStack.
Nova is one of the most common and important components of OpenStack. Compute is a controller that is used to handle virtualized environments' resources. It handles several virtual machines and other instances that perform computing tasks.
Nova is written in Python language. VMware, Xen, and KVM are the hypervisor technologies used, and this choice is contingent on OpenStack's version.
OpenStack Services which communicate with Nova
To ensure that Nova operates at its most basic level, certain OpenStack services are required. These services are:
Keystone: Firstly, Keystone authenticates and offers an identity for all OpenStack services. The first feature built on OpenStack is Keystone, and all projects, like Nova, are responsible for it.
Glance: It works to handle server images for your cloud. Therefore, it has the ability to upload compatible images of OpenStack via the repository of compute images.
Neutron: The physical or virtual networks that compute instances within your OpenStack cloud are given by Neutron.
Placement: Finally, Nova needs placement to track the inventory of resources to assist in selecting which resource provider would be the right option when building a virtual machine inside your OpenStack cloud.
To ensure optimum accessibility and performance, these additional OpenStack services closely interact with Nova.
The above diagram can be summed up in these functionalities :
Block Storage (Cinder)
This works in the traditional way of attaching and detaching an external hard drive to the OS for its local use. Cinder manages to add, remove, create new disk space in the server. This component provides the virtual storage for the VMs in the system. Conceptually, Cinder is similar in function to the EBS (Elastic Block Storage).
It is usually implemented in combination with other OpenStack services (e.g., Compute, Object Storage, Image, etc.). Cinder and Nova logical architecture are:
Without needing to think about costly physical storage systems or servers, Cinder users are able to reduce and expand their storage space significantly. In addition, by allowing users to use one code for each operation, Cinder simplifies code management. With reliability and ease of usage, Cinder can handle all the provisioning and eliminate consumers' needs.
Some of the goals of Cinder are :
Object Storage (Swift)
Object storage is used in order to store and recover arbitrary data in the cloud. In Swift, it is possible to store the files, objects, backups, images, videos, virtual machines, and other unstructured data. Developers may use a special identifier for referring the file and objects in place of the path, which directly points to a file and allows the OpenStack to manage where to store the files to the API.
For longevity, availability, and competitiveness, it is scalable and optimized. For storing unconstrained, redundant data, Swift is ideal. Since this is an object storage service, Swift enables an API-accessible storage option that can be used around the cluster for backups, data retention, or archives that are redundant.
Object Storage components are divided into the following key groups :
Let's see an example diagram for the OpenStack Object Storage :
Some Characteristics of OpenStack Object Storage are :
Shared File Systems (Manila)
It offers file-based storage to a VM. This component gives an infrastructure for managing and provisioning file shares. Manila uses a SQL based central database shared by all manila services in the system. The Manila service can operate in the configuration of a single node or multiple nodes.
Usually, Manila is deployed with other OpenStack resources, such as Compute, Image or Object Storage.
Following are the goals of shared file system service :
Manila offers the following set of services :
manila-api: It is an application for the Web Server Gateway Interface (WSGI), which verifies and guides requests via the shared file system service and also offers support to the OpenStack API.
manila-data: This service receives the requests, processes the data operations with long running times such as backup, copying, or share migration.
manila-scheduler: This service schedules and routes the requests to the shared file system services. To route requests, the scheduler follows configurable filters and weighers. The Filter Scheduler is the default and allows filters on items such as Availability Zones, Capacity, Capabilities, and Share Types. Manila-scheduler also allows custom filters.
manila-share: This service manages back-end systems in which have a shared file system. A manila-share service is capable of running in 1 of 2 modes, with or without the managing of shared servers.
The shared file system (Manila) contains the following set of components :
This component is used for networking in OpenStack. Neutron manages all the network-related queries, such as IP address management, routers, subnets, firewalls, VPNs, etc. It confirms that all the other components are connected properly with the OpenStack.
Neutron delivers NaaS (Networking-as-a-service) in a virtual computing environment. It has replaced the original API (Application Program Interface), called Quantum, in OpenStack. Neutron is managed by other OpenStack components such as Nova.
Networking has a service on the controller node, called the neutron server, including a lot of agents and plugins that use a messaging queue to communicate with each other. You can select the various agents you want to use, dependent on the type of operation.
Some features of Neutron:
There are the following neutron plugins :
The neutron architecture is very simple. It is fully based on agents and plugins.
This is the first component that the user sees in the OpenStack. Horizon is the web UI (user interface) component used to access the other back-end services. Through individual API (Application programming interface), developers can access the OpenStack's components, but through the Dashboard, system administrators can look at what is going on in the cloud and manage it as per their need.
At the core of its architecture and design, the Dashboard has many key values :
Core Support: Out-of-the-box provision for all core OpenStack projects.
Extensible: As a "first-class citizen", anyone can add a new component.
Manageable: The core codebase has to be easy to direct and should be simple.
Consistent: Throughout, visual and interaction paradigms are maintained.
Stable: A reliable Application program interface (API) with an emphasis on backward compatibility.
Usable: Providing an amazing interface that individuals want to use.
Horizon is based on the Django web framework for both users and administrators of an OpenStack cloud. It interacts with instances, images, volumes, and networks within an OpenStack cloud. Through Horizon, we can manage Nova, Glance, Neutron, and Cinder services within the OpenStack cloud.
The image below shows how the Dashboard is connected to all the OpenStack components. Notice that OpenStack with all seven core components is shown in this image :
Let's see the first screen. Once you open the Dashboard in your browser with the URL, you will get the following screen :
Identity Service (Keystone)
It is the central repository of all users and their privileges for the services they are using for OpenStack. This component is used to manage identity services like authorization, authentication, AWS Styles (Amazon Web Services) logins, token-based systems, and checking the other credentials (username & password).
There are the following functions which usually perform by Keystone:
There are the following set of services or components that are provided by identity service :
Server: It is a central system that offers authentication and authorization services using a REST interface.
Drivers: These are integrated with the centralized server. Drivers access the identity data in external repositories to OpenStack and may already exist in the infrastructure where OpenStack is deployed (Such as LDAP or SQL database servers).
Modules: In the location of the OpenStack service, which uses the Identity service, middleware modules run. These modules retrieve the requests for the service, extract user credentials and send them for authorization to the server. For integration between middleware modules and OpenStack services or components, the Python Web Server Gateway Interface is used.
Image Service (Glance)
The glance component is used to provide the image services to OpenStack. Here, image service means the images or virtual copies of hard disks. When we plan to deploy a new virtual machine instance, then glance allows us to use these images as templates. Glance allows virtual box (VDI), VMware (VMDK, OVF), Raw, Hyper-V (VHD) and KVM (qcow2) virtual images.
Glance contains a REST API from which you can query the metadata of a VM image and retrieve an actual image. It is central to IaaS (Infrastructure as a service).
The OpenStack Glance service contains the following components :
glance-api: For image detection, retrieval, and storage, glance-api accepts image API calls.
glance-registry: It stores, processes, and retrieves image metadata. Where items like size and type are included in the metadata.
Database: It is used to store image metadata, and according to your choice, you can select your database. MySQL or SQLite are used for most deployments.
Storage repository for image files: Standard file systems (or any file system installed on the glance-api controller node), HTTP, Object Storage, RADOS block devices, and VMware Datastore multiple repository forms are supported.
Metadata definition service: It is a normal API to meaningfully define your own custom metadata for suppliers, administrators, services, and users. Such metadata may be used for various resource types, such as images, artifacts, volumes, flavors, and aggregates.
It follows a client-server architecture. It provides the Rest API from which server request is made. Client requests are accepted via the Rest API and wait for authentication from Keystone. The Glance Domain Controller controls all internal processes, which are split into layers, and each layer performs its own tasks.
Image service architecture contains the following components :
Data Processing Service (Sahara)
The Sahara component offers a simple method to the users to preplanned Hadoop clusters by referring to the multiple options such as the Hadoop version, cluster topology and hardware details of nodes and some more.
Sahara is a data processing service, also called elastic map-reduce, which offers a platform for the provisioning and management of instance clusters using processing frameworks like Spark and Hadoop.
Following are the main features of Sahara :
Architecture of Sahara
As we know, cloud computing is getting wide in the last decade. Most of the biggest giants investing heavily to leverage the growing trend, such as Google, AWS, Microsoft, or Rackspace.
OpenStack is Open Source freely available with Apache 2.0 license and is backed by thousands of big technology companies. Over the years, it is kept on improving its popularity. Hundreds of enterprises are using it to build their own private cloud, and some are delivering support services. So, career-wise, in current business situations, I would suggest it is one of the best innovations to look for. The future calls for such a stable environment, and as a free architecture with some excellent long-term features, OpenStack will continue its legacy.
If you're looking for something that allows you to make better use of your current hardware, doesn't lock you into costly support contracts and has a wide global engineering community (over 100,000 members across 187 countries and increasing year on year by around 33 percent), then OpenStack could be the solution for you.
One thing is to remain there for secure cloud computing, with OpenStack getting a promising future for sure in at least the next ten years. OpenStack is a bundle of numerous cutting edge items that offer the best experience for cloud computing. So learning it and gaining experience in a few projects would not only make you a seasoned professional in the cloud but also help you develop skills in problem-solving.
OpenStack is one of the best environments in organizations for cloud computing. OpenStack's ease of linear scalability and open-source architecture have attracted many clients and enthusiasts of technology to come forward and contribute to development. This has only made OpenStack stronger over the years. OpenStack can be called for cloud computing, with all the benefits and endless modular functionality, as it proves to be an affordable option for the longer term.
Before proceeding with this tutorial, you must have a basic knowledge of cloud computing and the storage domain.
This tutorial is created for IT Managers and System Administrators who would like to learn the basics of OpenStack.
We assure you that you will not find any difficulty while learning our OpenStack tutorial. But if there is any mistake in this tutorial, kindly post the problem or error in the contact form so that we can improve it.