Choosing whether an AVM or Automated Valuation Model is best for your lending institution can be a huge decision. AVMs typically estimate “a property’s market value based on market, economic, and demographic factors.” It’ll either use a hedonic, index or blended model to provide a value, and you’ll need to have someone on your team, or a third party vendor, look over or regulate the responses from your AVM. Automation is not best for every lending institution or every real estate valuation, and it might not be right for you.
If you do choose this path, there are additional guidelines for selecting and using AVMs you must follow. AVMs on their own aren’t evaluations and need further examining for any property valuation.
- Pro Tip: Most AVMs can’t and don’t address physical condition. Make sure you aren’t missing any other key aspects in your appraisals.
Even when selecting an AVM, you must make sure you are reviewing the information provided by the AVM vendor and asking questions like this from the guidelines:
- How does the model “work”? Is it a hedonic model, an index model or a blended model?
- How did the vendor conduct performance testing in developing the model? What was the sample size and geographic level tested?
- On what data sources does the model rely? How often is the data updated? What about states where real estate sales data is not generally disclosed or easily accessed?
- How does the vendor conduct ongoing quality control testing and scoring?
Each model has inherent strengths and weaknesses, and the guidelines require an assessment of how the model performs for different property types such as condos, PUDs, single family detached residences, etc.
Before you start using an AVM, establish accepted minimum performance criteria and internal confidence score minimums. Regulations require you to make sure your AVM is meeting or exceeding this criteria and score. Therefore, either an employee or a third party vendor must ensure the AVM is providing a reliable value. If you choose to use a third party vendor, many have their own method of quantifying how reliable a model value is by using a rank ordering process, and this isn’t standardized. Plus, the regulators require that banks understand how the third party confidence score was derived and the extent to which it correlates to accuracy.
Whew! We understand that’s a lot of technicalities and regulations you must be abiding by to use an AVM. To make it easier, we wanted to provide excerpts from the guidelines to answer a few frequently asked questions on validation requirements. Here are the top 9 questions we see from lending institutions on AVMs:
1. Can the AVM vendor perform the validation?
An institution should not rely solely on validation representations provided by an AVM vendor. Individuals performing validation must be independent of the model development or sales function.
2. What qualifications do the individuals that perform the validation need?
An institution should ensure that persons who validate an AVM on an ongoing basis have the requisite expertise and training.
3. Can I engage a 3rd party?
Validation can be performed internally, or with the assistance of a third party, as long as the validation is conducted by qualified individuals.
4. What about independence from the loan production function?
An institution should ensure that persons who validate an AVM on an ongoing basis are independent of the loan production and collection processes.
5. I use multiple AVMs. Do they all have to be validated?
When selecting an AVM or multiple AVMs, an institution should perform a detailed validation of the model(s) considered during the selection process and document the validation process. If an institution uses more than one AVM, each AVM should be validated.
6. My AVM provider offers insurance or guarantees that support the AVM. Do I still have to validate it?
An institution should perform appropriate model validation regardless of whether it relies on AVMs that are supported by the value insurance or guarantees. If there are insurance or guarantee components of any particular AVM, the institution is responsible for understanding the extent and limitations of the insurance policy or guarantee, and the claim process and financial strength of the insurer.
7. Do I have to document the validation and have written policies and procedures? How often do they need to be updated?
When selecting an AVM or multiple AVMs, an institution should document the validation process. An institution should establish standards and procedures for independent and ongoing monitoring and model validation. An institution should document the results of its validation and audit findings. An institution should use [validation and audit findings] to analyze and periodically update its policies and procedures for an AVM(s) when warranted.
8. What are the minimum requirements for a validation procedure?
- [A]n institution should specify, at a minimum:
- Expectations for an appropriate sample size.
- Level of geographic analysis.
- Testing frequency and criteria for re-testing.
- Standards of performance measures to be used.
- Range of acceptable performance results.
9. What do the guidelines say about the depth and extent of validation? Are there any specific requirements in the guidelines for testing?
An institution should be able to demonstrate that the depth and extent of its validation process are consistent with the materiality of the risk and the complexity of the transaction. To ensure unbiased test results, an institution should compare the results of an AVM to actual sales data in a specified trade area or market prior to the information being available to the model. To assess the effectiveness of its AVM practices, an institution should verify whether loans in which an AVM was used to establish value met the institution’s performance expectations relative to similar loans that used a different valuation process.
[NOTE: Some agencies have published separate, detailed guidance on performing model validation, including the OCC. The OCC’s model validation guidance is referenced in a footnote in the appraisal guidelines’ AVM requirements addressing validation.]
Please note all answers are excerpts from the guidelines from Appendix B to the revised Interagency Appraisal and Evaluation Guidelines.