Implementing intelligent security using risk policies in Microsoft 365

There are two policies That we can enable in Microsoft 365 to help us find and block malicious users. 

• User risk policy
• Sign-in risk policy

What’s a user risk policy?

User risk is a calculation of the risk level that a user account has been compromised in Microsoft 365. The risk level is determined based on threat intelligence by reviewing normal behavior for the user. Some of the things that can be detected are the following:

• Anonymous IP address: These can come from users trying to log in to your tenant using a TOR browser or anonymous VPN.
• Unfamiliar sign-in properties: This is typically flagged when the user attempts to log in from a new device, location, or another behavior that is new to the user.
• Atypical travel: This user risk is flagged when a user signs in from a location that is different from the other recent sign-ins.
• Malicious IP address: A public IP address is considered malicious when there is a high failure rate of logins from that IP address.
• Suspicious inbox manipulation rules: Detected by Microsoft Defender for Cloud Apps this detection is triggered when a suspicious rule is created in the user’s inbox. Typically, the rule will automatically delete or hide messages.
• Impossible travel: Detected by Microsoft Defender for Cloud Apps this detection type is detected when the user logs in or performs another activity from two geographically distant locations. For example, the user logs in from the U.S.A. then ten minutes later logs in from Russia.

What’s a sign-in risk policy?

Sign-in risk policies show when a suspicious login happens in your Microsoft 365 environment. Microsoft 365 has a number of built-in sign-in risk policies.

• Activity from anonymous IP address: When a user logs in from an anonymous IP address
• Unfamiliar sign-in properties: This is typically flagged when the user attempts to log in from a new device, location, or another behavior that is new to the user.
• Impossible travel: Detected by Microsoft Defender for Cloud Apps this detection type is detected when the user logs in or performs another activity from two geographically distant locations. For example, the user logs in from the U.S.A. then ten minutes later logs in from Russia.
• Atypical travel: This machine learning method recognizes two sign-ins that originated from geographically different locations, one of which may be uncommon for the individual, based on previous behavior. The machine learning system takes into account the sign-in time slot, among other things.

Licensing Requirements

Sign-in risk policies and User risk policies require an Azure AD Premium P2 license for each user. The Azure AD Premium P2 license is bundled with Enterprise Mobility & Security E5 licenses as well.

Who can configure user risk policies and view users’ reports?

Global admins and Security admins can configure a user risk policy. Global admins, security admins, and security operators can view the user’s reports.

Different levels of risk

There are 4 different levels of user risk / sign-in risk. No risk, in which the user sign-in / activity appears to pose no threat. Low risk, which says there is a low chance the user has been compromised. Medium risk which says there’s a moderate chance the user is compromised. And finally, high risk. You guessed it, it means there is a high chance the user is compromised.

How to configure User Risk policies

1. Open the Azure Active Directory admin center > All services > Azure AD Identity Protection > User risk policy.

2. Click All users > Exclude > 0 users and groups selected. Click any users you want to exclude. Click Select.

3. Click Low and above (found under User risk). Click High > Done.

4. Click On (found under Enforce policy). Click Save.

How to configure Sign-in Risk policies

1. Open the Azure Active Directory admin center > Security > Identity Protection > Sign-in risk policy.

2. Click All users > Exclude > 0 users and groups selected. Click any users you want to exclude. Click Select.

3. Click Low and above (found under Sign-in risk). Click High > Done.

4. Click On (found under Enforce policy). Click Save.

Understanding the settings

Users

Under assignments, you can choose which users are included and which users are excluded. Exclusions take precedence so if you select All users then exclude a group of users then the group of users in the excluded list will be excluded from the policy.

User risk / Sign-in risk

Under the sign-in risk/user risk policy, you can set what level of sign-in risk will trigger the control. For example, in the setup above we configured everyone with a high level of risk.

Controls / Access

From here you can select what happens when the assignments are met. For example, if you select All users with a sign-in risk of high or above then you can decide if you want them to be blocked from signing in at all or require MFA.

If you require a user to validate their identity with MFA and they aren’t configured with MFA then the user will be blocked from signing in.

User experience

To see what the end-user will experience download the TOR Brower and attempt to login to your portal using any account that isn’t excluded from the policy. You should see a “Your sign-in was blocked”.

Once the account is blocked then even when they attempt to sign in without the triggering event they’ll receive “Your account is blocked”

How do you configure risk policies using conditional access policies?

So, blocking access or requiring password change / MFA isn’t enough. You need to get a little more detailed. You want to exclude your Hybrid Azure AD joined devices from being blocked. Let’s look at how to do that.

1. Open the Azure Active Directory admin center > All services > Azure AD Conditional Access > New policy > Create new policy.

2. Set the name to Sign-in Risk Policy.

3. Click 0 users or workload identities selected. Click All users.

4. Click No cloud apps, actions, or authentication contexts selected. Click All cloud apps.

5. Click 0 conditions selected > Not configured (found under Sign-in risk). Click Yes > High > Done.

6. Click Not configured (found under Device state (Preview) > Yes > Exclude > Device Hybrid Azure AD joined. Click Done.

7. Click 0 controls selected (found under Grant). Click Grant access > Require multi-factor authentication > Select. Click On (found under enable policy) > Create.

Investigate risk

Now you have policies in place how do we check to see if there are any malicious users getting blocked or any false positives where users are getting blocked but shouldn’t be?

There are three locations to review for risks: risky users, risky sign-ins, and risk detections.

Risky users

In the risky users' report administrators can view:

• What users are at risk.
• Details about what was detected
• Risk history of the user

View the risky users' report

1. Open Azure Active Directory admin center > Azure Active Directory > Security > Identity Protection > Risky users.

2. Click the username to review the report.

Risky sign-ins

The risky sign-ins report shows every sign-in for the last 30 days that had a low or higher risk. From the risky sign-ins report administrators can view:

• What sign-ins are labeled at risk.
• Detection type
• Device information (including OST, browser, and the compliance state of the device)
• Location information (including IP, country, and city)

View the risky sign-ins report

1. Open Azure Active Directory admin center > Azure Active Directory > Security > Identity Protection > Risky sign-ins.
2. Click the date/time of the sign-in you want to view.

Risk detections

The risk detections report allows administrators to view:

• Details about what was detected (including risk level and detection type)
• Location information (including IP, country, and city)

View the risk detections report

1. Open Azure Active Directory admin center > Azure Active Directory > Security > Identity protection > Risk detections
2. Click the date/time of the detection you want to view

Remediate risk and manage accounts

Now you’ve set up the policy and a user can’t log in! You’ve checked and they are blocked based on their own behavior. Maybe they logged into Microsoft 365 using the TOR browser. Not a problem. We can easily unblock the account. I do have one warning though. You can’t train the intelligent security policy to not block certain behavior. For example, even after you dismiss the user risk, they won’t be able to log in using the TOR browser. The TOR browser will still be blocked. If a user is required to perform some risky behavior, for example, they are required to use the TOR browser to login to Microsoft 365 then they’ll need to be excluded from the policy.

1. Go to Azure Active Directory admin center > Azure Active Directory > Security > Identity protection > Risky users. Click the user risk that you want to dismiss.

2. Click the ellipsis (…) > Dismiss user risk.

3. Click Yes.

4. Then go to Risky sign-ins

5. Click the sign-in you want to dismiss. Click the ellipsis (…) > Confirm sign-in safe.

6. Click Yes

Wait five to ten minutes and have the user try to log in again.

Whitelist IP addresses to help train the intelligent risk policies

One other thing you can do is flag your IP addresses as trusted. Once your organization's public IP addresses are flagged as trusted users will be less likely to be blocked in a false positive scenario.

1. Go to Azure Active Directory admin center > Azure Active Directory > Security > Risky sign-ins > Configure trusted IPs.

2. Click IP ranges location.

3. Enter a Name, for example, Main Office. Click Mark as trusted location. Click the + plus sign. Enter your public IP address + subnet mask. Click Add. Click Create.