Database Reference
In-Depth Information
Mean average precision at K
Mean average precision at K
(
MAPK
) is the mean of the
average precision at K
(
APK
)
metric across all instances in the dataset. APK is a metric commonly used in information
retrieval. APK is a measure of the average relevance scores of a set of the
top-K
documents
presented in response to a query. For each query instance, we will compare the set of
top-K
results with the set of actual relevant documents (that is, a ground truth set of relevant doc-
uments for the query).
In the APK metric, the order of the result set matters, in that, the APK score would be high-
er if the result documents are both relevant and the relevant documents are presented higher
in the results. It is, thus, a good metric for recommender systems in that typically we would
compute the
top-K
recommended items for each user and present these to the user. Of
course, we prefer models where the items with the highest predicted scores (which are
presented at the top of the list of recommendations) are, in fact, the most relevant items for
the user. APK and other ranking-based metrics are also more appropriate evaluation meas-
ures for implicit datasets; here, MSE makes less sense.
In order to evaluate our model, we can use APK, where each user is the equivalent of a
query, and the set of
top-K
recommended items is the document result set. The relevant
documents (that is, the ground truth) in this case, is the set of items that a user interacted
with. Hence, APK attempts to measure how good our model is at predicting items that a
user will find relevant and choose to interact with.
Note
The code for the following average precision computation is based on
https://github.com/
More information on MAPK can be found at
https://www.kaggle.com/wiki/MeanAver-
Our function to compute the APK is shown here:
def avgPrecisionK(actual: Seq[Int], predicted: Seq[Int], k:
Int): Double = {
val predK = predicted.take(k)
var score = 0.0
var numHits = 0.0
