Tuesday, December 4, 2012

Amazon VPC Essentials

Amazon Virtual Private Cloud (VPC) is a great way to setup an isolated portion of AWS and control the network topology. It is a great way to extend your data center and use AWS for burst requirements. In this post, I will list down the key areas that one needs to consider when working with VPC. This will help one decide the best architecture / solution that may fit the given requirement.

Except Cluster Compute Quadruple Extra Large instances all other Instance types (including Micro) are available within VPC. Of course, be sure to check which Instance types are available in a region. For example Second generation Standard On-demand instances are available only in US-East

Know which services and which features of a service are available within VPC
    • Dynamodb, ElastiCache, SQS, SES, SNS and CloudSearch are not yet available in VPC
    • Amazon RDS Micro Instances are not available in a VPC at this time - Update: AWS just announced (few minutes after posting this article) availability of Micro RDS Instances in VPC #paceofawsinnovation
    • RDS Instances launched in VPC cannot be accessed over the internet (through the end point). You will need a jump box (bastion) in the Public Subnet to connect to RDS or of course you can connect through the VPN from DC
    • DevPay AMIs cannot be launched within VPC
BGP is no longer a requirement for VPN connectivity from your devices. Static routing can be used for non BGP devices. But BGP is recommended since the liveness check that is performed is better. Also, each VPN conenctions has two tunnels for redundancy. If one of them goes down, the failover process is fairly simple in case of BGP

Customer Gateway Compatibility
The following customer gateway devices are tested and known to be working with AWS VPC. Make sure you are using one of these devices at the Customer Gateway side. Other devices may be compatible as well
  • Static Routing
    • Cisco ASA 5500 Series version 8.2 (or later) software
    • Cisco ISR running Cisco IOS 12.4 (or later) software
    • Juniper J-Series Service Router running JunOS 9.5 (or later) software
    • Juniper SRX-Series Services Gateway running JunOS 9.5 (or later) software
    • Juniper SSG running ScreenOS 6.1, or 6.2 (or later) software
    • Juniper ISG running ScreenOS 6.1, or 6.2 (or later) software
    • Microsoft Windows Server 2008 R2 (or later) software
    • Yamaha RTX1200 router
  • Dynamic Routing using BGP
    • Astaro Security Gateway running version 8.3 (or later)
    • Astaro Security Gateway Essential Firewall Edition running version 8.3 (or later)
    • Cisco ISR running Cisco IOS 12.4 (or later) software
    • Fortinet Fortigate 40+ Series running FortiOS 4.0 (or later) software
    • Juniper J-Series Service Router running JunOS 9.5 (or later) software
    • Juniper SRX-Series Services Gateway running JunOS 9.5 (or later) software
    • Juniper SSG running ScreenOS 6.1, or 6.2 (or later) software
    • Juniper ISG running ScreenOS 6.1, or 6.2 (or later) software
    • Palo Alto Networks PA Series running PANOS 4.1.2 (or later) software
    • Vyatta Network OS 6.5 (or later) software
    • Yamaha RTX1200 router
IP Ranges
When setting up a VPC you are essentially fixing the network of the VPC. And if the VPC requires VPN connectivity (as in most of the cases), care should be taken to choose the IP range of the VPC and avoid any IP conflicts.
  • Assess the IP ranges used at the customer gateway side
  • If the customer gateway side has more than one data center which already have VPN connectivity between them, assess all the data centers
  • Also check the MPLS link between the data centers and the IP range used by the MPLS provider
The above two points (Device Support & IP Range) becomes the critical point in setting up the VPC. If the chosen IP ranges result in IP conflicts, this will jeopardize the entire network architecture and can potentially bring down existing systems in the customer data center

Public and Private Subnets
The VPC network can be divided further in to smaller network segments called as Subnets. Any VPC will have at least one Subnet
  • You can setup a Public Subnet which will have internet connectivity. Instances launched within a Public Subnet will have both outbound and inbound (through EIP) internet connectivity through the Internet Gateway attached to the Public Subnet
  • Private Subnets are completely locked down. They do not have internet connectivity by default
  • Create number of Public and Private Subnets depending upon your architecture. Place all public facing servers such as web servers, search servers in the public subnet. Keep DB servers, cache nodes, application servers in the private subnet
Security Groups
VPC Security Groups are different from normal EC Security Groups. With EC2 Security Groups you can control the ingress into your EC2 Instance. With VPC Security Groups, you have the option to control both inbound and outbound traffic. When something is not accessible you have to check both inbound and outbound rules set in the VPC Security Group

ELB Security Group 
When you launch an ELB within VPC, you have the option to specify a VPC Security Group to be attached with the ELB. This is not available for ELB launched outside VPC in normal EC2. With this additional option, you can control access to specific ELB ports from specific IP sources. On the backend EC2 Instances' Security Group, you can allow access to the VPC Security Group that you associated with the ELB

Internal ELB 
When you launch an ELB within VPC, you also have additional option to launch it as an "Internal Load Balancer". You can use an "Internal Load Balancer" to load balance your application tier from the web tier above. Please refer to my earlier article on Internal Load Balancer

NAT Instance 
By default the Private Subnets in a VPC do not have internet connectivity. They cannot be accessed over the internet and neither can they make outbound connections to internet resources. But let's say you have setup a database on an EC2 Instance in the Private Subnet and have implemented a backup mechanism. You would want to push the backups to Amazon S3. But the Private Subnet's cannot access S3 since there is no internet connectivity. You can achieve it by placing  a NAT Instance in the VPC
  • Through NAT Instance outbound connectivity for Private Subnet Instances can be achieved. The Instances will still not be reachable from the internet (inbound)
  • You need to configure the VPC Routing Table to enable all outbound internet traffic for the Private Subnet to go through the NAT Instance
  • AWS provides a ready NAT AMI (ami-f619c29f) which you can use to launch the NAT Instance
  • You can have only one NAT Instance per VPC
  • Since you can have only one NAT Instance per VPC, you need to be aware that it becomes a Single Point Of Failure in the architecture. If the architecture depends on the NAT Instance for any critical connectivity, it is an area to be reviewed
  • And you are limited by the bandwidth availability of a single NAT Instance. So do not build architecture that will have internet bandwidth requirements from the Private Subnet with NAT
Elastic Network Interface
This is a powerful feature of VPC which lets you solve many problems around licensing, hosting multiple SSL websites, etc...
  • You can attach multiple ENI's per Instance essentially creating multiple private IPs and possibility of attaching multiple EIP's per Instance
  • One interesting result of attaching multiple ENI's per Instance is this: previously you are used to launching EC2 Instances in a Subnet. Now EC2 Instances can span across Subnets. For example 
    • You can have two ENIs - one attached to the public Subnet and other attached to the private Subnet
    • You attach a Security Group to each of these ENIs. The Security Group of public Subnet ENI will allow access on port 80 from the internet but no access on, let's say, port 22
    • The other Security Group attached to the Private Subnet ENI will allow port 22 access from the VPN gateway essentially securing the Instances more
  • If there are products that have licenses tied to the MAC address, you can now set the MAC address to the ENI and have the EC2 Instance derive the them from ENI
Private IP and Elastic IP
  • You can fix the Private IP of the Instance when you launch Instances within the VPC.  Let's say, you plan to launch multiple tiers (web servers and search servers) within a Subnet and have multiple Instances for each tier. Make a logical allocation of IP ranges for each tier. For example, to for web servers. Whenever you launch any Web Server Instance use one of the private IPs from the logical list. This will help in management of the potentially vast number of Instances
  • Private IP of an Instance do not change if you stop and start the Instance
  • If you attach an Elastic IP to the Instance, the EIP does not change if you stop and start the Instance


Santhosh said...


nice article, short and covers all the vpc essentials.

I am having a situation because of cost constraints and current design restrictions I have to setup a FTP server on the Database instance running inside a private subnet. I have a NAT server running in the public subnet. Will I be able to upload files to it ? (because you said the instance will still not be reachable from the internet )

If that's not possible, will having two ENIs and using one of the ENIs for FTP be a good idea ?



Raghu said...

What is the purpose of the FTP server? And why it has to be inside the Private Subnet? If you are placing something in the private subnet, you will obviously set it up to secure it and access it only within the VPC network or through the VPN. Please let me know the use case.

And if you are running the FTP server on the Private Subnet and if there is a NAT Instance, still you will not be able to reach the FTP server through internet.

Using two ENIs can be looked at but I would like to hear your use case before I would suggest something like that

Santhosh said...

Hey Raghu,

Thanks, I will try out the ENI approach.

More than a use-case, its sort of a restriction that I am not allowed to mess with the current setup.

Customers need to upload their data onto the data warehouse server on a daily basis. ETL tasks are run on the uploaded data during the non-peak hours.
The data warehouse server is a m2.4xlarge server, we thought we can use the capacity of the server after EOD to run the ETL tasks on the DB server itself than adding a new server.

We did think of using S3 instead of FTP to the DB server, but HTTPS is way slower compared to SFTP.

Redshift along with datapipeline will be very useful for us going forward, I guess.

Raghu said...

You can go with the ENI option but you are opening up the database server to the internet. But you can secure it this way:

Essentially you are looking at two ENIs being attached to the server - on having an Elastic IP and the other having just the private IP. You got to do the following
1. Attach a security group to the first ENI and allow only SFTP access
2. Attach another security group to the second ENI and open it up to only the security groups of other Instances within the VPC
3. On the first security group open up the SFTP port to only a specific IP (like your data center / corporate office public IP)

This way you are at least opening up the SFTP to a known IP address

Krishna said...

Good summary of all the features. I have a few diagrams illustrating the subnets and availability zones at my blog
After deploying the VPC architecture, I have become a firm believer. There is a slight learning curve, but the folks at AWS have made it simpler enough.

Raghu said...

Great slides. Yes, VPC will be the way to go in AWS. EC2 will become classic. 8-9 months back many services (such as ELB,RDS) weren't even available in VPC. But all of them are available now in VPC. And every new service/feature that AWS rolls out is being rolled out for VPC as well. VPC is maturing fast.

Anand Kanade said...

Hi Raghu!,

We are looking for solution
Here is our requirement.
1.We want to keep our web server's in private subnet.
2.We want to configure LB on these web servers so that we can access LB from internet.
3.We want to access certain outside web sites from these web server instances ( os update, clam updates). vpc setup/settings for same.

P.S. We don't want to assign public IP on our web servers.

We tried NAT instance for outgoing from private subnet instance. It works but then we are not able to access LB from outside.
As we have to configure NAT instance as gateway on router.

Pl. let us know solution fro our requirement.


Raghu said...

Hi Anand,

You should have the LB in your public subnet so that it can be accessed over the internet via the internet gateway. If you are using ELB, launch the ELB within VPC and attach the web server Instances - but do not choose "Launch as an Internal Load Balancer". If you are using some other software load balancer (like HaProxy on EC2), make sure you launch them in the public subnet. Hope this helps. Let me know if you are still facing any issues.