Use VCE Exam Simulator to open VCE files
This Chapter covers following Topic Lessons
This Chapter covers following Lab Exercises
In this chapter we will add Monitor, Log Analytics, Advisor & Service Health to the topology. We will Install Diagnostic Agent (DA) on VM VMFE1. We will connect VM VMFE1 to Log Analytics by installing Microsoft Monitoring Agent (MMA) on VM VMFE1.
For Monitoring of resources we will enable or create Activity Logs, Diagnostic Logs, Alerts and Action Groups.
Azure includes multiple services that individually perform a specific role or task in the monitoring space. Together, these services deliver a comprehensive solution for collecting, analyzing, and acting on telemetry from your application and the Azure resources that support them.
The figure below shows a conceptual view of the components that work together to provide monitoring of Azure resources.
Note: We can monitor resource Diagnostic Settings and Activity logs through resource Dashboard or through Monitor Dashboard.
Azure Monitor provides centralized dashboard for viewing Logs, metrics & alerts for Azure resources.
Azure Monitor provides mini-dashboards for Metrics, Activity Log, Diagnostic logs, alerts, Service Health, Network Watcher & Application Insights etc. Data can be exported to Log Analytics and Power BI for further Analysis.
All data collected by Azure Monitor fits into one of two fundamental types - Metrics and Logs.
Metrics are numerical values that describe some aspect of a system at a particular point in time. They are lightweight and capable of supporting near real-time scenarios.
Logs contain different kinds of data organized into records with different sets of properties for each type. Telemetry such as events and traces are stored as logs in addition to performance data so that it can all be combined for analysis.
In Azure Portal click Monitor in Left pane> Monitor Dashboard opens> In the left pane you can see tabs for Alerts, Metrics, Activity Log, and Diagnostic settings etc.
Note: Readers are requested to explore options in the left pane.
Activity Logs provide data about the operations on a resource from the outside. The Activity Log reports control-plane events for your subscriptions. For Example Azure Activity log will log an event when a virtual machine is created or a logic app is deleted. But any Activity performed by virtual Machine will not be reported by Activity Log.
You can monitor Activity log for Compute as well as non-compute Resources.
Activity Log is a Platform level Service. You don’t require any agents to be installed. Events can be seen in Azure Portal. Events Logs can be exported to Azure Storage, Event Hubs, Power BI and OMS Log Analytics. You can create alerts on Events generated in Activity Log.
In Azure Portal click Monitor in Left pane> Monitor Dashboard opens> Click Activity Log. This will report control plane events for all the resources . Note : By default Activity dashboard will show logs for last 6 Hours. To see Logs for different duration you need to select duration from Timespan dropdown box.
To see activity log for particular resource such as Virtual Network “VNETCloud”
Click Add Filter 2 times>Select Resource RGCloud>Select Resource VNETCloud.
Go to Virtual Network “VNETCloud” Dashboard> Click Activity Log in left Pane.
Azure resource-level diagnostic logs are logs emitted by a resource about the operation of that resource. Diagnostic logs provide insight into operations that were performed within that resource itself, for example, getting a secret from a Key Vault.
Difference between Activity and Diagnostic Logs: Activity Logs provide data about the operations on a resource from the outside (the "control plane"). Diagnostics Logs are emitted by a resource and provide information about the operation of that resource (the "data plane").
You can monitor Diagnostic log for Compute as well as non-compute Resources.
Diagnostic log is a Platform level Service. You don’t require any agents to be installed and nor you require any Azure Level service to be created. You just need to enable diagnostic logs for the resource.
The Figure below shows the Architecture of Diagnostic Log.
Diagnostic Log is a Platform level Resource which logs Events generated by the resource. Events Logs can be seen in Azure Portal.
Diagnostic Logs can be streamed to Event Hubs for ingestion by a third-party service or custom analytics solution such as PowerBI. You can Analyze logs with OMS Log Analytics.
In Monitor Dashboard click Diagnostic settings in left pane> Right pane shows diagnostic logs status for all the Azure resources in the subscription.
In Monitor Dashboard click Diagnostic settings in left pane> In Right pane scroll down and click RSVCloud> From here you can enable Diagnostic setting for Recovery Services Vault RSVCloud. RSVCloud was created in Exercise 76, Chapter 7.
Click Turn on diagnostics>Diagnostic Setting blade opens>Enter a name, Select Storage account and/or Event Hub and/or Log Analytics and select logs as per your requirement and click OK>Click Save.
For this lab I select Storage Account sastdcloud and for Log I selected AzureBackupReport.
Note: Backup Reports data will be fully available in the storage account after 24 hours from configuration.
Back reports in Storage Account sastdcloud as shown below.
In this exercise we will enable Diagnostic Logs for Network Security Group NSGCloud. NSGCloud was created in Exercise 11, Chapter 1. We will analyse NSG Diagnostic Logs with Log Analytics in Exercise 158 in this chapter.
In Monitor Dashboard click Diagnostic settings in left pane> In Right pane Click NSGCloud>Click + Add diagnostic setting>Diagnostic Setting pane opens>Enter a name>Select Send to Log Analytics>Select the Logs> Click Save>Close the pane after you get notification about successful updation.
Guest OS-level/Extended Metrics are collected by installing diagnostics agent on the Azure virtual machine. If you don’t enable Guest level monitoring then only Standard Metrics or Host level metrics are available.
Guest OS-level diagnostic logs capture data from the operating system and applications running on a virtual machine. Guest OS-level diagnostic logs collect following types of Metrics and Logs:
Performance counters
Application Logs
Windows Event Logs
.NET Event Source
IIS Logs
Manifest based ETW
Crash Dumps
Customer Error Logs
Go to VM VMFE1 Dashboard>Click Diagnostic settings in left pane>In Right pane click Enable Guest Level Monitoring.
After enabling you can see following in overview screen.
Click on Performance Counters>Custom>< You can now see OS level/VM level counters>Click save (Not shown).
Checking VM level Performance Counters . In VM VMFE1 dashboard click metrics in left pane> In Resource select VMFE1> In metric Namespace select Guest (Classic) > In Metric Dropdown box you can see VM level counters are available now.
Note1 : Metric Namespace Guest (Classic) is not available if you don’t enable Guest Level Monitoring.
Note 2: Readers are requested to see Click other options in Diagnostic settings like Logs, Crash Dumps, Sinks and Agent. Click Basic Option also.
As an exercise to readers go to VMFE2 dashboard>Click Metrics in left pane>In Right pane Click Metric Namespace Dropdown box> You will not see the Guest Option.
Metrics are numerical values that describe some aspect of a system (CPU or Memory utilization etc.) at a particular time. They are lightweight and capable of supporting near real-time scenarios. Metrics are collected at regular intervals whether or not the value changes.
They're useful for alerting because they can be sampled frequently, and an alert can be fired quickly with simple logic.
Percentage CPU metric will collect processor utilization from a virtual machine every minute. You have the option to configure and fire an alert on the metric such as when one of those collected values exceeds a defined threshold.
Metric are Key value pairs. Metric for Percentage CPU and Network Throughput are shown below:
Percentage CPU
Timestamp Metric Value
8/9/2017 8:14 70
Network Throughput
Timestamp Metric Value
8/9/2017 8:15 1,141.4 Kbps
There are three fundamental sources of metrics collected by Azure Monitor. All of these metrics are available in the metric store where they can be evaluated together regardless of their source.
Platform metrics are created by Azure resources and give you visibility into their health and performance. Each type of resource creates a distinct set of metrics without any configuration required.
Application metrics are created by Application Insights for your monitored applications and help you detect performance issues and track trends in how your application is being used. This includes such values as Server response time and Browser exceptions.
Custom metrics are metrics that you define in addition to the standard metric that are automatically available. Custom metrics must be created against a single resource in the same region as that resource.
Note 1: Readers are advised to see more metrics through Metric Dropdown box.
Note 2 : You can also access Metric through Monitor dashboard.
In Monitor Dashboard click Metrics in left pane> In Metric Pane Click Add Metric>Click Resource Box and in Resource Group Select RGCloud, In Resource type select Storage Accounts and then select Storage Account sastdcloud and click apply> In Metric Drop Box select Used Capacity>You can now see real time chart for last 24 Hours.
An action group is a collection of notification preferences (Action Types) defined by the user. Various Alerts types use Action Groups when the alert is triggered.
Email/SMS/Push/Voice
Azure Function
Logic App
Webhook
ITSM
Automation Runbook
In this exercise we will create Action group with Action type Email. We will use this Action Group with Alerts in Alert exercise.
In Azure Portal click Monitor in Left pane> Monitor Dashboard opens>Click Alert in left pane> In Right pane you can see Manage Actions.
Click Manage action in right pane>Action Group pane opens>Click +Add Action Group>Add Action group blade opens>Give a name and short name> Select System created Resource group> Enter action name>Select Email/SMS/Push/Voice in Action type>Click Edit details> Email/SMS/Push/Voice detail pane opens> Enter a name>Select Email check box and enter email id>Click OK>Click Ok.
Note1 : You can add Multiple action types in a group.
Note 2 : Readers are advised to check all action types and detail pane which opens for that Action type.
You can now see Action group created in Action Group pane.
Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to identify and address issues before the users of your system notice them.
For Example you can create an alert on Virtual Machine Metric that if CPU utilization goes above 70% then send an email or start an additional instance.
Alert Rule: The alert rule captures the target and criteria for alerting.
Target Resource : A target can be any Azure resource such as virtual machine, a storage account, a virtual machine scale set, a Log Analytics workspace, or an Application Insights resource.
Signal : Signals are emitted by the target resource and can be of several types - Metric, Activity log, Application Insights, and Log.
Criteria: Criteria is combination of Signal and Logic applied on a Target resource.
Action : A specific action taken when the alert is fired and is specified in Action Group.
In this lab we will create Alert on Metric (Percentage CPU) with a criteria that if CPU utilization goes above 70% in VM VMFE1 hen notify through an e-mail. We will use Action group created in Exercise 153 for notification
In Monitor Dashboard Click Alert in left pane>Alert Pane opens.
In Alert pane click + New Alert Rule> Create rule blade opens>Click select under Resource>Select a resource blade opens>In Resource type drop down box select Virtual Machines>Under resources select VM VMFE1>Done.
Note : You have option of accessing Alert from VM VMFE1 dashboard also.
Under Condition Click Add> Configure Signal Logic Blade opens.
Select Percentage CPU>Percentage Platform blade opens>Scroll down> In Threshold Box enter 70>Click Done
Under Actions Click Add> Select an Action Group blade opens>Select action group Created in Exercise 153>Add>
In Alert Details Enter a name for Alert Rule, Select Severity Level>Select Yes for Rule creation>Enter a description>Click Create Alert Rule.
In Alert pane click Manage alert Rules>You can see the alert rule created> If required you can edit the rule also.
In Azure Portal go VM VMFE1> Under Monitoring Click Alerts in left pane>Alert Pane opens as shown below.
Log Analytics is a service in Operations Management Suite (OMS) that helps you collect and analyze data generated by resources in your cloud and on-premises environments.
It gives you real-time insights using integrated search and custom dashboards to readily analyze millions of records across all of your workloads and servers regardless of their physical location.
Log Analytics has 2 components " OMS Workspace & Monitoring Agent. The combined solution of Log Analytics service and OMS repository is known as OMS Workspace. OMS repository is hosted in the Azure cloud.
Microsoft Monitoring Agent is installed on the connected source. Data is collected into the repository from connected sources.
Figure below shows Log Analytics collecting and analyzing data generated by resources in Azure, on-premises and other Clouds.
Connected Sources can be on-premises or Cloud Resources. All Resources which you have created in your Subscription will appear in Log Analytics Dashboard under various Data Sources. You can add following Connected Sources in Log Analytics Services.
Data sources are configured on connected sources. Data sources can be IIS Logs, Performance Counters, Syslog, windows security events, windows firewall log, Network Security group.
Design Nuggets: You can create multiple workspaces in Azure Subscription. Workspaces are independent of each other and that data collected from each workspace cannot be viewed in another workspace.
Log Analytics collects data from managed resources into a central repository. This data could include events, performance data, or custom data provided through the API. Once collected, the data is available for alerting, analysis, and export.
Log Analytics includes a powerful query language to extract data stored in the repository. The result of the query can be viewed in following ways:
Dashboard: You can view the result of the query in Log Analytics Dashboard.
Export: You can export the results of any query to analyze it outside of Log Analytics. You can schedule a regular export to Power BI which provides significant visualization and analysis capabilities.
Log Search API. Log Analytics has a REST API for collecting data from any client. This allows you to programmatically work with data collected in the repository or access it from another monitoring tool.
You can create Alerts on the Log search data. In addition to creating an alert record in the Log Analytics repository, alerts can take the following actions.
Email. Send an email to proactively notify you of a detected issue.
Runbook. An alert in Log Analytics can start a runbook in Azure Automation. This is typically done to attempt to correct the detected issue. The runbook can be started in the cloud in the case of an issue in Azure or another cloud, or it could be started on a local agent for an issue on a physical or virtual machine.
Webhook. An alert can start a webhook and pass it data from the results of the log search. This allows integration with external services such as an alternate alerting system, or it may attempt to take corrective action for an external web site.
In this exercise we will monitor IIS server running in Azure VM VMFE1. There are 4 steps involved in this: Creating Log Analytics workspace, Add Connected source, Add data source and Query IIS Log data using log search.
Step 1: Create Log Analytics workspace (Log Analytics service + OMS Repository)
Click + Create a resource > Management Tools > Log Analytics> create Log Analytics workspace blade opens>For name I entered LACloud>For Resource Group I selected create new resource group with name mlogs> East US 2 Location >Click ok>After Validation is successful close the pane.
Accessing Log Analytics workspace LACloud dashboard > Click All resources in left pane>All Resources blade opens>Scroll down and you can see LACloud.
Click LACloud (Fourth row in above figure)> LACloud dashboard opens as shown below> in left pane we scrolled down to see Workspace Data sources options.
In Log Analytics workspace dashboard in left pane scroll down and under Workspace Data sources click Virtual Machines> Right pane shows the Virtual Machine VMFE1 and the status of Log Analytics Connection. Which in this case is not connected.
In right pane click VM VMFE1> Connect Blade opens> Click Connect. This will install Log Analytics Agent VM Extension> After a minute it gets connected. Status will show This workspace. Close the connect blade.
Note: Log Analytics VM Extension installs Microsoft Monitoring Agent on the machine.
In Log Analytics workspace dashboard click Advanced Settings in left pane> In Advanced Setting pane click Data>Click IIS logs> Select Collect IIS Log files>Save> Close the Advance Settings pane.
Click Windows Performance Counters>In right pane you can see various options for performance counters. We are not adding performance counters.
Readers are advised to scroll down and see all options for counters.
Access Default Website on VMFE1 using Chrome and Tor Browser.
In Log Analytics workspace dashboard click Logs in left pane> Logs Query Pane opens as shown below.
In Right pane enter Query W3CIISLog | summarize count() by cIP > Click Run>You can see the Client IPs which have accessed default website.
Management solution packs have Pre Built rules and Algorithms that perform analysis leveraging Log Analytics services. Management solutions are added to Log Analytics Workspace.
Management solutions are available both from Microsoft and partners.
Below are some of the Management solutions which can be added to Log Analytics service.
Active Directory Health Check: Active Directory Health Check solution assesses the risk and health of your server environments (Domain Controllers) on a regular interval. The solution provides a prioritized list of recommendations specific to your deployed server infrastructure.
AD Replication Status: The AD Replication Status solution pack regularly monitors your Active Directory environment for any replication failures.
Alert Management Solution: The Alert Management solution helps you analyze all of the alerts in your Log Analytics repository.
Network Performance Monitor (NPM): The Network Performance Monitor management solution is a network monitoring solution that monitors the health, availability and reachability of networks.
Network Security Group analytics solution: Network Security Group analytics management solutions collect diagnostics logs directly from Network Security Groups for analyzing them in Log analytics.
Container Monitoring Solution: Container Monitoring Solution shows which containers are running, what container image they’re running, and where containers are running. You can view detailed audit information showing commands used with containers.
Key Vault Analytics solution: Azure Key Vault solution in Log Analytics reviews Azure Key Vault logs.
Office 365 management solution: Office 365 management solution allows you to monitor your Office 365 environment in Log Analytics.
Service Fabric Analytics: Identify and troubleshoot issues across Service fabric Clusters.
Service Maps: Automatically discovers and Maps servers and their dependencies in real-time. Service Map automatically discovers application components on Windows and Linux systems and maps the communication between services. It also consolidates data collected by other services and solutions to assist you in analyzing performance and identifying issues. Service Map shows connections between servers, processes, and ports across any TCP-connected architecture, with no configuration required other than the installation of an agent.
SQL Server Health Check: SQL Health Check solution assesses the risk and health of your SQL Server environments on a regular interval.
Update Management: Identifies and orchestrates the installation of missing system updates. This solution requires both Log Analytics and Automation account.
Change Tracking: Tracks configuration changes across your servers. This solution requires both Log Analytics and Automation account.
Antimalware Assessment: OMS Antimalware Assessment solution helps you identify servers that are infected or at increased risk of infection by malware.
Azure Site Recovery: Monitor’s Virtual Machine replication status for your azure Site Recovery Vault.
IT Service Management (ITSM) connector: Connects Log Analytics with ITSM Products such as servicenow.
In this Exercise we will add Management Solution Network Security Group Analytics to Log Analytics. It will show details of NSG flows. In Exercise 148 we enabled Diagnostic Logs for Network Security Group NSGCloud and sent the logs to Log Analytics workspace.
In Log Analytics workspace dashboard click Workspace Summary in left pane.
Click + Add in Workspace Summary blade>Management Tools Pane opens> In Recommended Solutions Click more>Recommended blade opens>Scroll down and select Azure Network Security Group Analytics .
Azure Network Security Group Analytics blade opens>Click Create.
Create new Solution Blade opens> Click create (Not Shown). Close the pane after Validation is successful.
After Deployment is succeeded go to Log Analytics Dashboard and click Solutions in left pane> You can see NSG Analytics Solution is added.
Wait for 1-2 hours and then Click on AzureNSGAnalytics as shown in previous figure> AzureNSGAnalytics dashboard opens>Click on Summary and you can see the NSG Analytics.
Click on the summary Box and you can see the details of NSG Flows.
In this exercise we will just demonstrate the steps on how to install MMA in On-Premises Server or VM.
From your on-premises VM open browser and log on to Azure Portal.
Go to Log Analytics Workspace dashboard> Click Advanced Settings in left pane>Connected Sources> Select Windows or Linux Servers as per your requirement>In right pane download the agent on your Windows or Linux Machine> Note down Workspace ID and Primary Key. This will be required to register the server with Log Analytics Workspace during installation.
Azure Advisor is a personalized recommendation engine that provides proactive best practices guidance for optimally configuring your Azure resources.
Azure Advisor gives recommendation to optimize across following four different areas.
High Availability
Performance
Security
Cost
All recommendations accessible in one place on the Azure portal. Azure Advisor is a free service.
It analyzes your resource configuration and usage telemetry. It then recommends solutions to help improve the performance, security, and high availability of your resources while looking for opportunities to reduce your overall Azure spend.
In Azure Portal click Advisor in Left pane> Advisor Dashboard opens. It shows recommendation in 4 areas - High availability, Performance, Security and Cost.
Click on High Availability Tab and you can see there are 3 Recommendations.
Click on Cost Tab and you can see their is 1 Recommendation. It says that you can save Indian Rupees 2360 If we delete Public IP which is currently not associated with any running Azure Resource. Recall that we created this Public IP in Chapter 1.
Click on Security Tab and you can see there are 7 Recommendations.
Click on the recommendation and its shows recommendation available from Security Center.
Azure Service Health provides status of Azure services which can affect your business critical applications. It also helps you prepare for upcoming planned maintenance. Azure Service Health alerts you and your teams via targeted and flexible notifications.
Service Health tracks following three types of health events that may impact your resources:
Go to Monitor Dashboard>Click Service Health Tile in left pane> Service Health Dashboard Opens> currently there are no Service issues.
Click on Planned maintenance in left pane> No events are scheduled.
Click on Health advisories in left pane> No advisories are found.
Note: Readers are advised to Check Resource Health Tab in left pane.
In this Exercise we will create email Alert on Service Health events " Service Issues, Planned maintenance and Health Advisories.
In Service Health Dashboard click Health alerts in left pane.
Click +Create Service Health Alert in Right pane>Add Rule Blade opens> Under Alert target Select Services & Region as per your req>Click Add under Actions to Select Action Group created in Exercise 153>Under Alert details Give a name and Select mlogs in Resource Groups> click Yes to enable rule and Click Create Alert Rule.
Top Training Courses
LIMITED OFFER: GET 30% Discount
This is ONE TIME OFFER
A confirmation link will be sent to this email address to verify your login. *We value your privacy. We will not rent or sell your email address.
Download Free Demo of VCE Exam Simulator
Experience Avanset VCE Exam Simulator for yourself.
Simply submit your e-mail address below to get started with our interactive software demo of your free trial.