How to create an AWS S3 bucket with automation using Terraform and Jenkins pipeline on an AWS ec2 server.
Today, I will show how you can deploy an s3 bucket with Terraform on AWS ec2 instance using git and creating a CI-CD Pipeline through GitHub and Jenkins Pipeline. We will walkthrough the whole process from scratch.
The steps will be as follows:
Create an AWS ec2 instance on AWS.
Install Java JDK on Linux Server and update the server.
Install debian stable binary and jenkins.
Install Terraform on Linux server.
Start Jenkins and configure the Admin User.
Manage plugins and add Terraform.
Setup Jenkins pipeline and write a groovy script.
Create a webhook between GitHub and Jenkins.
Debug the issues in the pipeline and check if the s3 bucket has been created.
Step 1. Create AWS ec2 instance.
Go to aws.amazon.com and search EC2 and click on Launch an instance.
Select Ubuntu 20.04 which is available in the Free Tier of Amazon
Create a new keypair and give it a name of your choice. This key is very important as it will help in connecting and configuring your server.
Step 2. Connect your terminal to the server and start configuring.
Click on your instance and click on the connect button.
Copy the ssh code here and open it on your Terminal
cd into your directory and then execute the following command.
Now you have connected your terminal to your AWS ec2 Ubuntu Server.
Let's execute the following command step by step and later I will explain their meanings.
sudo apt-get update
sudo apt install default-jre -y //java jdk
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key |
sudo apt-key add -
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
/etc/apt/sources.list.d/jenkins.list'
Update the server.
sudo apt-get update
Now install jenkins on server
sudo apt-get install jenkins
Step 3. Write terraform code in Visual Studio Code.
Make a main.tf file and write the following code for s3 bucket.
provider "aws" {
region = "ap-south-1"
}
resource "aws_s3_bucket" "b" {
bucket = "myawsbucket123455544"
acl = "private"
tags = {
Name = "My bucket"
Environment = "DevJKT"
}
}
Now push this code using git using the following commands.
Step 4. Start Jenkins and configure it.
Start Jenkins using the following command.
sudo service jenkins status
Now go to your Chrome Browser where your instance would be running and click on that instance and search for details. You will get an instance address something like this, go to that address link and add PORT as :8080.
You will get a result like this on the browser.
Click on install suggested plugins and it will start installing the required plugins as of now.
Next you need to create an admin user for Jenkins , give out the basic information and then it will ask for an Administrator password.
Write this code on your server terminal and you will get your password.
Next you will land on the Homepage of Jenkins after entering the password. Go to your profile and then configure for getting the access token key. You need to generate one. Copy this and save it somewhere in the notepad.
Step 5. Connect your GitHub with Jenkins using webhooks.
Go to the settings page of GitHub repo and then go to Webhooks. Create a new webhook and in the url section give out your address like this:
http://{YOUR_INSTANCE_ADDRESS}:8080/github-webhook/
SECRET Here you need to paste the access token key that you have saved in the notepad which we got from Jenkins Configuration. Click on add webhook.
Step 6. Let's Do it finally
Go to your Jenkins homepage on the server and click on Manage Plugins.
Search Terraform there and click on Install without restart.
Now comeback to your Manage Jenkins page and click on Global Tools Configuration. We need to add Terraform here.
Now you have successfully added Terraform to your Jenkins Globally.
Next Step:
- Go to your homepage and click on New Item. Give it a name , choose Pipeline and add.
- Now in the configuration step, select the Git grooving script and then in pipeline script write the following script.
This script is for setting up the pipeline and writing out commands so that the pipeline will function properly step by step.
pipeline {
agent any
tools {
terraform 'terraform'
}
stages {
stage ("checkout from GIT") {
steps {
git 'https://github.com/JaguarsCodehub/terraform-jenkins.git'
}
}
stage ("terraform init") {
steps {
sh 'terraform init'
}
}
stage ("terraform fmt") {
steps {
sh 'terraform fmt'
}
}
stage ("terraform validate") {
steps {
sh 'terraform validate'
}
}
stage ("terrafrom plan") {
steps {
sh 'terraform plan '
}
}
stage ("terraform apply") {
steps {
sh 'terraform apply --auto-approve'
}
}
}
}
Next Click on Build Now button which you will get it on your left side of the HomePage.
I will be sharing some screenshots of the successful pipeline setup and also that the s3 bucket will be created in AWS using the name which you provided in the terraform main.tf file.
Here you can see, after I commit something to GitHub the pipeline automatically starts and the deployment process turns on !
The S3 bucket has been created successfully.
This is a CI-CD Pipeline where Continuous Integration and Continuous Deployment happens which means when you write a new code in the file and after your commit, The pipeline will automatically deploy the code to Jenkins and then the terraform script will run and it will create a s3 bucket automatically.
Terraform is so much powerful as it helps to configure Cloud resources and services directly through code and scripts and it speeds up the development process with the cloud. It also skips the whole setup thing which developers used to do previously where everything was done through the User Interface of these Cloud Providers.
You can create tons of services and configure a hell lot of things using Terraform it skips the whole part of walkthrough of creating a service, You can see how easily an s3 bucket was created with just one terraform file.
Thanks for reading this blog till the end and comment your thoughts, also more blogs would be coming soon which will be DevOps specific and also various other topics.
Regards, Jyotindra Tavanoji.