Annalisa opinions bring together the BEANs for the performance rates of each option on each criterion with the criteria importance weights of the person owning the decision. For some criteria the person is the best source of the BEANs, since they are the expert, e.g. on how difficult it is to visit a clinic or to use a particular medication route. For other criteria we will want to derive the estimates from relevant studies. Studies are conventionally evaluated on a hierarchy of evidential quality that reflects both the study design and its implementation in the research. The scale runs from a meta-analysis of high-quality randomised controlled trials to an anecdote (\'what my mate in the pub thinks\'). An explicit or implicit cut-off is often applied on the scale to separate what is regarded as \'real\' evidence from that regarded as not meeting an \'acceptable\' standard of evidential quality.

Since research is very unequally funded and conducted across relevant patient-important criteria, a DecisionAL will almost invariably contain criteria for which the \'evidence quality\' varies from good to poor by such conventional standards, often falling below proposed cut-offs or thresholds.

The simple, quality unadjusted Annalisa opinion treats the BEANs for every criterion as equal, irrespective of their quality.
The Quality Adjusted Opinion (QAO) reflects the opinion of the producers on the quality of the evidence underlying each criterion.

How is the QAO produced?

The DecisionAL producers draws on conventional grading techniques to arrive at an overall characterisation of the Evidential Quality of the Ratings (EQR) for each criterion. This is a number between 0 (worthless as evidence, lacking all quality) to 1 (of the highest possible quality). For each criterion the EQR has to reflect any differences in evidential quality across all the options, for that criterion.
If the quality for each criterion was assessed as 1, the quality-adjusted scores and hence the QAO, would be the same as those in the unadjusted Annalisa. If the the same EQR is assigned to every criterion, but not 1, the quality-adjusted scores would all be lower - but no different relative to one another.

The QAO is produced by adjusting the person\'s weightings for the various criteria by the EQRs for them.

In our (purely hypothetical) example both quantity and quality of life are weighted 50% and the unadjusted opinion favours more exercise because of the ratings.

If the EQR for quantity of life was 80% (very strong evidence), then its Quality-Adjusted Weight (QAW) is 50% x 80% or 40%...provisionally. We say provisionally, because the QAWs for all criteria must add to 100%, so the final QAW for any one criterion will depend on that of all the other QAWs. So let’s imagine the producers assessed the evidence on quality of life as weak and gave it an EQR of 20%, meaning its provisional QAW would be 50% x 20% or 10%. The two provisional QAWs being .4 and .1, the final normalised ones are .8 and .2. And better nutrition now has the higher score, whereas the unadjusted opinion was in favour of more exercise.

Note that the QAO will depend on the criterion importance weightings of the person. While the producers have provided a consensus EQR for each criterion, these are applied to the individual respondent\'s weightings. So if one person gave low weight to a criterion with a low EQR, such as 20% to quality of life, the QAO will differ very little from the unadjusted one. On the other hand, if they give a high weight to it, this greater importance could overcome the weak evidence. In a revised example, with quantity of life weighted 20% and quality of life 80%, the evidential quality adjustment is insufficient to flip the opinion in favour of better nutrition. 

Whereas at lower weights for quality of life (such as 70%) it will do so.