Understanding FAR, FRR, and EER in Auth Systems

Have you ever wondered how systems determine whether to grant or deny access, and how they balance the risk of false acceptance with usability? This tutorial explores the fundamental concepts behind evaluating authentication systems or classification models using False Acceptance Rate (FAR), False Rejection Rate (FRR), and Equal Error Rate (EER). These metrics are essential for assessing the balance between usability and security in auth systems. Gaining a good understanding of these terms can greatly enhance both theoretical insights and practical application in designing reliable machine learning systems.

What is False Acceptance Rate (FAR)?

The False Acceptance Rate measures how frequently a system incorrectly grants access to an unauthorized individual. For instance, in a bank’s facial recognition system, if an unauthorized person manages to gain entry due to the system misidentifying them, it would count as a false acceptance. It provides an indication of the system’s vulnerability to breaches caused by overly lenient decision-making.

Let represent the total number of unauthorized attempts, and the number of false acceptances. Then, FAR can be expressed as:

FAR = (Number of false acceptances / Total number of fraudulent attempts) × 100

Consider a fingerprint scanner where 5 out of 100 unauthorized attempts are accepted. This might happen due to imperfections in the scanning process, such as low-quality fingerprints or environmental factors like dirt on the scanner. The implications of these false acceptances include potential security breaches, where unauthorized individuals gain access to sensitive areas or data.

This means that 5% of unauthorized users were mistakenly granted access.

A high FAR signals security risks, which is unacceptable in scenarios such as financial transactions or access control for secure facilities. Reducing FAR is crucial for safeguarding against potential breaches.

What is False Rejection Rate (FRR)?

The False Rejection Rate quantifies how often a system mistakenly denies access to legitimate users. This rate is critical to system usability because frequent rejections can lead to frustration and decreased trust in the system. When legitimate users are consistently denied access, it not only impacts their experience but also reduces the likelihood of the system being widely adopted in user-centric applications. This metric reflects the system’s usability and impacts the overall user experience. For example, in a healthcare system using facial recognition to verify patient identities, a high FRR could lead to critical delays in accessing medical records or services, causing frustration and potential harm to the user.

Let the total number of legitimate attempts be , and the number of false rejections be . The formula for FRR is:

FRR = (Number of false rejections / Total number of legitimiate access attempts) × 100

In a facial recognition system, if 2 out of 100 authorized users are denied access:

This shows that 2% of legitimate users were incorrectly rejected.

A high FRR frustrates users, particularly in customer-facing applications, where ease of access is as important as security. Balancing FRR with security is essential for maintaining trust in the system. For instance, in sensitive applications like banking or healthcare, excessive rejections can erode confidence and impede critical operations, highlighting the need for a well-calibrated approach. Reducing FRR is essential for enhancing system usability.

What is Equal Error Rate (EER)?

The Equal Error Rate represents the point at which the FAR and FRR are equal, marking a critical threshold for evaluating system performance. This balance is significant as it reflects the system’s ability to optimize both security and usability without overly compromising one for the other. It serves as a benchmark metric, offering a holistic view of the system’s performance by balancing security and usability.

To determine the EER, follow these steps:

1. Adjust the decision threshold to observe variations in FAR and FRR.

2. Locate the intersection point where FAR equals FRR.

3. Record the error rate at this threshold—this is the EER.

This metric is significant because it enables comparison across different systems and algorithms. It also serves as an optimization target, as a lower EER indicates better overall system performance, particularly in high-security environments. EER is widely used to compare the effectiveness of different systems or algorithms. 

A lower EER indicates better overall performance, making it a desirable goal in high-security environments.

Applications in Auth Systems

The metrics sucha s FAR, FRR and ERR are used in auth systems like fingerprint or facial recognition in the following manner:

• FAR highlights the risk of unauthorized access.

• FRR reveals usability challenges faced by legitimate users.

• EER helps in setting an optimal balance between the two for the system.

EER can be visualized on a Detection Error Trade-off (DET) Curve or a Receiver Operating Characteristic (ROC) Curve, which graphically represent the relationship between false positives and false negatives at various thresholds. These curves are useful for identifying the optimal operating point and understanding how adjustments to thresholds affect system performance., where the intersection of FAR and FRR lines is identified. These tools provide clarity on the threshold’s impact on system performance.

Conclusion

Understanding FAR, FRR, and EER is essential for building reliable authentication and classification systems. These metrics are not only central to optimizing system performance but also have broader applications in fields such as fraud detection, cybersecurity, and user authentication, where balancing security and usability is critical. These metrics provide a foundation for achieving the right balance between security and user experience, ensuring your system operates effectively in any real-world application.