This architecture requires a connection to your local data center, either using a VPN gateway or an ExpressRoute connection. Typical applications for this architecture are

• Hybrid applications in which workloads are executed partly locally and partly in Azure.
• Infrastructure that requires differentiated control of data traffic coming from a local data center into a virtual Azure network.
• Applications that need to monitor outgoing traffic. Monitoring is a requirement of many commercial systems, often due to government regulations, and can help prevent the disclosure of private information.

Use role-based access control in Azure (Azure Role-Based Access Control, Azure RBAC) to manage the resources in your application. Consider creating the following custom roles:

• A DevOps role with permissions to manage the infrastructure for the application, to deploy the application components and to monitor and restart VMs.
• A central IT administrator role for managing and monitoring network resources.
• An IT security administrator role to manage the secure network resources, such as the firewall.

The IT administrator role should not have access to the firewall resources. Access should be restricted to the IT security administrator role.

Azure resources such as VMs, virtual networks and load balancing modules can be easily managed by grouping them into resource groups. Assign Azure roles to each resource group to restrict access.

It is advisable to create the following resource groups:

• A resource group that contains the virtual network (without the VMs), the network security groups and the gateway resources for connecting to the local network. Assign the central IT administrator role to this resource group.
• A resource group that contains the VMs for the Azure Firewall instance and custom routes for the gateway subnet. Assign the IT security administrator role to this resource group.
• Separate resource groups for each virtual spoke network containing the load balancer and the VMs.

To allow inbound traffic from the Internet, add a Destination Network Address Translation (DNAT) rule to the Azure Firewall.

• Destination address = public IP address of the firewall instance
• Translated address = private IP address within the virtual network

Use tunnel enforcement for all outgoing Internet traffic through your local network by using a site-to-site VPN tunnel and use Network Address Translation (NAT) for routing to the Internet. This design prevents the accidental disclosure of any confidential information while allowing all outbound traffic to be inspected and monitored.

Do not completely block internet traffic from the resources in the subnets of the spoke network. Blocking traffic will prevent these resources from using Azure PaaS services that rely on public IP addresses, such as VM diagnostic logging, downloading VM extensions, and other functions. Azure Diagnostics also requires components to have read and write access to an Azure Storage account.

Check that outgoing Internet traffic is properly tunneled. If you are using a VPN connection with the routing and RAS service on a local server, use a tool such as WireShark.

These considerations are based on the pillars of the Azure Well-Architected Framework, a set of principles that can improve the quality of workloads. For more information, see Microsoft Azure Well-Architected Framework.

Efficient performance

Performance efficiency is the ability of your workload to efficiently scale to meet user requirements. For more information, see Overview of the "Performance efficiency" pillar.

For more information about VPN Gateway bandwidth limits, see Gateway SKUs. For larger bandwidths, you can consider upgrading to an ExpressRoute gateway. ExpressRoute provides up to 10 Gb/s bandwidth with less latency than a VPN connection.

For more information about the scalability of Azure gateways, see the scalability sections at:

• Implementing a secure hybrid network architecture with Azure and local VPN
• Implementing a secure hybrid network architecture with Azure ExpressRoute

For detailed information on managing virtual networks and NSGs on a large scale, see Tutorial: Creating a protected hub-and-spoke network.

Security offers protection against deliberate attacks and the misuse of your valuable data and systems. You can find more information under Overview of the "Security" pillar.

This reference architecture implements several levels of security.

Route all local user requests through an Azure Firewall

The user-defined route in the gateway subnet blocks all user requests that do not originate from the local network. The route forwards permitted requests to the firewall. The requests are forwarded to the resources in the virtual spoke networks if they are permitted by the firewall rules. You can add more routes, but make sure that they do not inadvertently bypass the firewall or block management traffic for the management subnet.
 

Using NSGs to block/pass traffic to virtual network networks

Traffic to and from resource subnets in virtual spoke networks is restricted by the use of NSGs. If you have a need to extend the NSG rules to allow wider access to these resources, weigh these needs against the security risks. Each new inbound communication path represents an opportunity for accidental or intentional disclosure of data or corruption of applications.

AVNM allows you to create baselines for security rules that can take precedence over network security group rules.Security management rules are evaluated before NSG rules and have the same nature as NSGs, with support for prioritization, service tags and L3-L4 protocols. This allows central IT to enforce basic security rules that are independent of additional NSG rules from Spoke-VNet owners. To facilitate a controlled rollout of security rule changes, you can use AVNM's Deployments feature to securely release the breaking changes of these configurations in the hub-and-spoke environments.
 

DevOps-Access

Use Azure RBAC to restrict the operations that DevOps can perform at each level. Use the least privilege approach when granting permissions. Log all management operations and perform regular monitoring to ensure that all configuration changes are based on planning.

Cost optimization

Cost optimization is about finding ways to reduce unnecessary expenditure and improve operational efficiency.