Differences

This shows you the differences between two versions of the page.

--- guide:cloud [2020/08/19 08:26]
lphukungoane [Cloud Resources]
+++ guide:cloud [2025/10/12 12:31] (current)
sfebruary [CHPC Cloud Resources]
@@ Line 1: / Line 1: @@
-======Cloud Resources======
+======CHPC Cloud Resources======
-The **CHPC Cloud System** is a CentOS-based cloud computing infrastructure powered by [[https://www.openstack.org/ | Openstack]] and open source storage software [[https://ceph.io/ | Ceph]]. Openstack provide services like server virtualization, block & object storage systems, software defined networking, an image repository, a web based dashboard and authentication.You can read more about [[https://docs.openstack.org/ussuri/ |Openstack and its various projects]].
+    *[[guide:cloud:overview|Overview]]
+    *[[guide:cloud:start|Getting Started]]
+    *[[guide:cloud|User Guide]]
+       *[[guide:cloud:advanced|IAC (Advanced)]]
+    *[[guide:cloud:practices|Best Practice and Policies]]
+    *[[guide:cloud:glossary|Glossary of Terms]]
+    *[[guide:cloud:faq|FAQs]]
-The **Centre for high performance Computing (CHPC)** hosts an OpenStack cloud built on Supermicro server and storage systems, using the Supermicro TwinPro servers to provide 320cores/640threads (2.50 - 3.90GHz) and over 3TB DDR4 memory providing some 9GB RAM per core and all in just 4U of rack space. It is connected through Mellanox 100GB ethernet networking to Supermicro Ultra and Supermicro Simply Double Servers, providing a CEPH Storage cluster with over 1.5PB (1500TB) of mechanical disk storage and more than 220TB of flash storage, offering resources such as:
-  * **Compute Resource:**  provides you with access to a large set of CPU & RAM and are provisioned on a per Instance basis.
+======How can I use an OpenStack cloud?======
-  * **Storage Resource:** Block & Object storage functionality based on the open source software 'Ceph'. Ceph provides a scalable and highly available storage solution across many commodity servers.
-  * **Networking Resource:** provides 10Gbps Internet connectivity, 10Gbps inter-host and 240Gbps inter-rack network connectivity using Openstack's Software Defined Networking (SDN).
+As an OpenStack cloud end user, you can provision your own resources within the limits set by cloud administrators.
+The examples in this guide show you how to perform tasks by using the following methods:
-^   CPU cores  ^ Memory    ^ Storage   ^Networking(Shared) ^ Object Storage (raw) ^Block Storage ^
+  * **OpenStack dashboard:** Use this web-based graphical interface, code named [[ https://opendev.org/openstack/horizon/ | horizon]], to view, create, and manage resources.
-|  3452        | 78 TB     | 4.4 PB    |  10 Gbps          | 403 TB               | 403 TB       |
-| :::          | :::       | :::       |  :::              |:::                   | :::          |
+  * **OpenStack command-line clients:** Each core OpenStack project has a command-line client that you can use to run simple commands to view, create, and manage resources in a cloud and automate tasks by using scripts.
+----
-**Fees for using the CHPC cloud system**
-The cloud resources are allocated on per project basis, then several instances are provisioned based on the available resources allocated to the project. The billing is calculated by allocation not by percentage of utilization. The current billing fees are calculated as follows:
-  * R0.23 per vCPU hour.
-  * R0.14 per GB hour of vMemory(vRAM).
-  * R0.0017 per GB hour of storage (volumes, images and object storage).
+===== Introduction - Volumes=====
+Openstack volumes are logical block devices that can be attached to a single instance to provide a persistent location for data storage. Unlike a virtual machine's local disk which is destroyed along with the VM, volumes are decoupled from instances so that they may be attached and reattached to different instances. This allows you to create & destroy instances as you see fit while maintaining a persistent store for data.
-The Cloud is now open to a broader community of researchers. Apply today and start using our resources.
+**1. Create Volume.**
-A new project account creation on the Cloud user database can be requested through the [[ https://openstackusers.chpc.ac.za/ | application process]]. Upon acquiring cloud user database account and submitting a project on the CHPC cloud user database, we will send you an email with further instructions.
-Cloud users are expected to:
+From the Openstack dashboard, click **Compute > Volumes > Create Volume**, fill out the form by providing a meaningful name, description and size. Once this is done, click **‘Create Volume’**. Your volume will be created and you can move on to the next step, ‘Attach a volume to an instance’.
-  * Make appropriate use of Cloud resources and use good social behavior.
+{{ :guide:screenshot_2020-08-20_volumes_-_openstack_dashboard_1_.png?direct&900 |}}
-  * Do not share private ssh keys or login information.
+----
-  * Cite the Cloud in any papers describing research.
+**2. Attach a volume to an Instance.**
-  * Submit tickets to the [[ https://users.chpc.ac.za/helpdesk/ | CHPC support ticketing system]] when encountering technical issues not covered by the Cloud support documentation.
+From the Openstack dashboard, click **Compute > Volumes** and then on the Actions drop down for the volume you want to attach to an Instance, click **‘Manage Attachments’.**
+{{ :guide:screenshot_2020-08-20_volumes_-_openstack_dashboard_2_.png?direct&900 |}}
+//From the Manage Volume Attachments form, select the Instance you wish to attach to and click **‘Attach Volume’**. This will dynamically attach the volume as a new logical block device to your instance. Move on to the next section of identifying the block device from within your Instance.//
-Already a Cloud user? Login to the [[https://openstack.chpc.ac.za | Cloud Console]] and get started with running your analysis.
 ----
-======How can I use an OpenStack cloud?======
-As an OpenStack cloud end user, you can provision your own resources within the limits set by cloud administrators.
+**3. Identify A volume within Instance.**
-The examples in this guide show you how to perform tasks by using the following methods:
+SSH into your instance, su to root and run **‘lsblk’**
-  * **OpenStack dashboard:** Use this web-based graphical interface, code named [[ https://opendev.org/openstack/horizon/ | horizon]], to view, create, and manage resources.
+//**‘vda’** is the first disk of your Instance and contains one partition **‘vda1’** which contains the operating system. Any volumes attached to your Instance will be named **‘vdb, vdc, vdd’ & etc**. Identify the new disk by checking the size and comparing that to the size you created in the Openstack dashboard. Now that you have identified the logical block device from within the Instance you can move on to Formatting.//
-  * **OpenStack command-line clients:** Each core OpenStack project has a command-line client that you can use to run simple commands to view, create, and manage resources in a cloud and automate tasks by using scripts.
 ----
+**4. Formatting the Volume.**
- ======OpenStack dashboard======
+Prerequisite: Install xfsprogs
-As a cloud end user, you can use the OpenStack dashboard to provision your own resources within the limits set by administrators. You can modify the examples provided in this section to create other types and sizes of server instances.
+As root: On Ubuntu based systems
-For your convenience, we offer these short instructions that you should follow in order to start a new instance, while also recommending you read the official user guide to familiarize with more advanced functionalities.
+    apt-get install xfsprogs -y
-. First please open [[https://openstack.chpc.ac.za]] in your browser and login using the credentials provided.
+**Format using xfs**
+We are using **xfs** in this example because it is widely supported and supports extremely large volume sizes (9 exabytes)
-{{ :guide:screenshot_2020-06-17_login_-_openstack_dashboard.png?direct&400 |}}
+    mkfs.xfs -f /dev/vdc
 ----
-. Launching a virtual machine, You have to accomplish a number of steps.
+**5. Mounting a Volume.**
-a) Provide the initial hostname for the instance, the availability zone where it will be deployed, and the instance count. Increase the Count to create multiple instances with the same settings.:
+Create a directory to mount the volume (example ‘data1’), mount the volume and then verify the mount.
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_4_.png?direct&4000 |}}
+As root:
-b) Select the instance source, the template used to create an instance. You can use an image, a snapshot of an instance (image snapshot), a volume or a volume snapshot (if enabled). You can also choose to use persistent storage by creating a new volume.
+      "mkdir data1
+       mount -t xfs /dev/vdc /data1
+       df -hT /data1"
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_3_.png?direct&4000 |}}
+----
+**6. Re-attaching the volume to a different Instance.**
-// For this example. The instance is launched from the image source and  a new volume is created with the instance. The allocated **ubuntu-18 image** is selected from the list of available images and is going to be an Operating System (OS) for the instance/s created. //
+**a) Unmount the volume.**
-----
+As root, use the command **"umount"** to unmount the directory **data1**
+    umount /data1
+//NB Remove or comment out the line added in **/etc/fstab** if applicable.//
-c) Decide on the flavor needed when launching the instance, Flavors manage the sizing for the compute, memory and storage capacity of the instance.
+**b) Detach a volume.**
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_5_.png?direct&4000 |}}
-// For this example. The instance is launched with **m1.tiny flavor** selected from the list of available flavors. The m1.tiny flavor will allocate the instance with 2x vCPU, 5GB of vRAM and 20GB of root disk.
+From the Openstack dashboard, click **Compute > Volumes** and then on the Actions drop down for the volume you want to attach to an Instance, click **‘Manage Attachments’**.
-In order to carefully manage our compute resources, Only allocate what you are expecting to use, This is the total compute resources allocated to the instance created.//
+{{ :guide:screenshot_2020-08-20_volumes_-_openstack_dashboard_3_.png?direct&900 |}}
+From the Manage Volume Attachments form, Click **‘Detach Volume’** for the instance you wish to detach the volume from.
+----
+**7. Re-attach Volume.**
+Follow the steps at the beginning of this guide to attach the volume to a desired instance. **DO NOT Format the volume from the new instance.** Skip that step and go directly to mounting. The volume is now mounted on a different Instance and the data is preserved.
 ----
+**8. Extend Volume (Grow)**
-d) Select the network that your instance will connect to. The network provide the communication channels for instances in the cloud.
+Volumes can also be extended in size. In order to Extend the volume it must be unmounted and detached from an Instance.
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_6_.png?direct&4000 |}}
+     Note: Volumes cannot have their size reduced.
+{{ :guide:screenshot_2020-08-20_volumes_-_openstack_dashboard_4_.png?direct&900 |}}
+Enter the new size of the volume and click **Extend Volume**.
-//For this example. The instance is launched with **demo-vxlan** network selected from the list of available networks. The network provide the communication channel for instance/s in the cloud.//
+Re-attach the volume to your Instance, and remount the volume from within the instance. You will need to grow the xfs filesystem before it will make the new space available.
+As root:
+     xfs_growfs /dev/vdc -d
+For more details, visit the openstack official pages:[[https://docs.openstack.org/mitaka/user-guide/dashboard_manage_volumes.html|Manage Volumes]]
 ----
+===== Upload and manage images =====
+A virtual machine image, referred to in this document simply as an image, is a single file that contains a virtual disk that has a bootable operating system installed on it. Images are used to create virtual machine instances within the cloud. For information about creating image files, see the [[https://docs.openstack.org/image-guide/ | OpenStack Virtual Machine Image Guide.]]
-e) Select the security groups to launch the instance in.
+**1. Upload an image**
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_7_.png?direct&4000 |}}
+Follow this procedure to upload an image to a project:
-//For this example. The instance is launched with **Default security group**  selected from the list of available security groups. The security groups are set of restrictive rules defined for the IP space, The selected default security group allows only ssh(22) and HTTPS(443). Openstack allows security groups editing after the instance has started, so additional changes can be made later if the exact requirements are not clearly defined.//
+   a) Log in to the dashboard.
+   b) Select the appropriate project from the drop down menu at the top left.
+   c) On the Project tab, open the Compute tab and click Images category.
+   d) Click Create Image.
+Then Create An Image dialog box appears
-----
+{{ :guide:screenshot_2020-08-20_images_-_openstack_dashboard.png?direct&900 |}}
-f) Select an existing key pair, import a key pair, or generate a new key pair. A key pair allows you to SSH into your newly created instance.
-{{ :guide:file.jpeg?direct&4000 |}}
-----
+//Select **"Create Image"**, After you have filled all the required information.//
-g) Optionally, You can customize your instance after it has launched using the options available here. "Customization Script" is analogous to "User Data" in other systems.
-{{ :guide:screenshot_2020-08-19_instances_-_openstack_dashboard_9_.png?direct&4000 |}}
 ----
-h) Finally, click the Launch button. It will take a few minutes for the instance to start, depending on how large the base image you chose was, and if it was already cached or not on the physical server where the VM will be scheduled to start.
+**How to deploy windows on openstack**
-----
+Windows Image for OpenStack: For most Linux distributions, it is possible to download a generic QCOW2 file for the purposes of running instances in a cloud environment. However, Microsoft does not provide such instances for any Cloud platform other than Azure.
-======How can I access the instance or virtual machine ?======
+Since Microsoft Windows is still a major component in many IT infrastructures, there may be a need to run Windows-based instances on OpenStack.
-  * **OpenStack dashboard:** Use console,
+To upload Windows image, We suggest that you follow this tutorial for [[https://superuser.openstack.org/articles/how-to-deploy-windows-on-openstack/|Preparing windows images]] before uploading to the openstack environment.
-{{ :guide:screenshot_2020-08-19_testt_wvm_-_openstack_dashboard.png?direct | 4000}}
+       Note: CHPC does not provide licenses, Appropriate licenses for any software run on the service by
+       users must be sought and applied by the users, including any licenses that may be required to run
+       instance operating systems.
 ----
-  * **ssh into virtual machine:** When you started the VM and specified the name of your key pair, its public part was automatically added to the default user’s//** “~/.ssh/authorized_keys”**// file, which will allow you SSH access into it.
-From your laptop, first change the permissions for the private SSH key, so your SSH client doesn’t complain when you will try to use the key:
-    chmod 400 path_to_private_key
-Then, using the desired SSH client application, run:
-    ssh -i path_to_private_key ubuntu@X.X.X.X
+===== Access and security for instances =====
+    work in progess...!
-//NB The X.X.X.X denotes to the IP address (use public IP allocated to the instance).// //**ubuntu** is the default username for ubuntu images and// //**centos** for CentOS images//

wiki.chpc.ac.za

User Tools

Site Tools

Differences

Page Tools