Total Pageviews

Sunday, 21 August 2011

Micro- and Macro-average of Precision, Recall and F-Score

I posted several articles explaining how precision and recall can be calculated, where F-Score is the equally weighted harmonic mean of them. I was wondering- how to calculate the average precision, recall and harmonic mean of them of a system if the system is applied to several sets of data.

Tricky, but I found this very interesting. There are two methods by which you can get such average statistic of information retrieval and classification.

1. Micro-average Method

In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the statistics. For example, for a set of data, the system's

True positive (TP1)= 12
False positive (FP1)=9
False negative (FN1)=3

Then precision (P1) and recall (R1) will be 57.14 and 80

and for a different set of data, the system's


True positive (TP2)= 50
False positive (FP2)=23
False negative (FN2)=9

Then precision (P2) and recall (R2) will be 68.49 and 84.75

Now, the average precision and recall of the system using the Micro-average method is

Micro-average of precision = (TP1+TP2)/(TP1+TP2+FP1+FP2) = (12+50)/(12+50+9+23) = 65.96
Micro-average of recall = (TP1+TP2)/(TP1+TP2+FN1+FN2) = (12+50)/(12+50+3+9) = 83.78

The Micro-average F-Score will be simply the harmonic mean of these two figures.

2. Macro-average Method

The method is straight forward. Just take the average of the precision and recall of the system on different sets. For example, the macro-average precision and recall of the system for the given example is

Macro-average precision = (P1+P2)/2 = (57.14+68.49)/2 = 62.82
Macro-average recall = (R1+R2)/2 = (80+84.75)/2 = 82.25

The Macro-average F-Score will be simply the harmonic mean of these two figures.

Suitability
Macro-average method can be used when you want to know how the system performs overall across the sets of data. You should not come up with any specific decision with this average.

On the other hand, micro-average can be a useful measure when your dataset varies in size.

246 comments:

  1. Thanks for the article, just a minor typo: second to last paragraph: Micro rather than macro.

    ReplyDelete
    Replies
    1. Thanks a lot, Yahya. I corrected it. But the typo is in the last paragraph. If your dataset varies in size, micro-average is the useful tool.

      Delete
    2. hi, thank you for the article, please i have a general question related to learning algorithms. I found this expression in an article : The training data for the SVM classi ers are highly unbalanced as the proportion of positive training web pages ranges from 2% to 18%.
      What do we mean by unbalanced data? and by positive/negative training?
      ReplyDelete

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. I think there's a problem with your macro-average precision calculation. the denominator should be the sum of true positives and false positives. The micro average for precision and recall are the same. See here:
    http://metaoptimize.com/qa/questions/8284/does-precision-equal-to-recall-for-micro-averaging

    ReplyDelete
  4. @Pallika. The scenarios are different. In my case, one classifier is applied on two different datasets. The number of instances are different in two datasets. But the example you provided is classifier performance for three different class labels. So the sum of positive and negative is always 27. The two macro and micro evaluation is different.

    ReplyDelete
  5. Hi..Thank you for the article. I did some exparimentation on Text classification which consists 7 (labled documents prepared on my own)ctegories of documents.I did too much stemming and i got average precision 57%, micro precision=macro precision=1,micro recall=macro recall=0.please suggests me something to improve results(like bad training set,light stemming, threshold value and threshold step size modification, k if classifier is knn,how to improve recall for better F1 score).
    Thanks.

    ReplyDelete
  6. Good article.... if you explain what is micro and macro average in 3 lines it will be more hepful for readers...

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Really sweet, thank you! That was hell of a good explanation! :)

    ReplyDelete
  9. Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking. Data Science online Training Hyderabad

    ReplyDelete
  10. Thank you for sharing your article. Great efforts put it to find the list of articles which is very useful to know, Definitely will share the same to other forums.
    Data Science Training in chennai at Credo Systemz | data science course fees in chennai | data science course in chennai quora | data science with python training in chennai

    ReplyDelete
  11. Thank you very much for sharing your article

    ReplyDelete
  12. I like this post and thanks to share. Post is very nice, If you having any errors from your antivirus service call our toll free number 1-888-883-9839 (USA)
    For more Information visit our websites:- https://www.antivirussupporthelplinenumber.us/

    ReplyDelete
  13. sklearn implements 'macro' f1 as average of the two f1's and not harmonic means of the average precision and recall.

    f1 = (f1_1 + f1_2) / 2

    Check: https://github.com/scikit-learn/scikit-learn/blob/51a765a/sklearn/metrics/classification.py#L1098

    ReplyDelete
  14. Excellent Blog. I really want to admire the quality of this post. I like the way of your presentation of ideas, views and valuable content. No doubt you are doing great work. I’ll be waiting for your next post. Thanks .Keep it up! Kindly visit us @Luxury Boxes
    Premium Packaging
    Luxury Candles Box
    Earphone Packaging Box
    Wireless Headphone Box
    Innovative Packaging Boxes
    Wedding gift box
    Leather Bag Packaging Box
    Cosmetics Packaging Box
    Luxury Chocolate Boxes

    ReplyDelete
  15. thanks for your information really good your work web design company in velachery

    ReplyDelete
  16. thanks for your information really good and very nice web design company in velachery

    ReplyDelete
  17. the blog is very use for me .keep sharing like this.web design company in velachery

    ReplyDelete
  18. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information.web design company in velachery

    ReplyDelete


  19. Soma pill is very effective as a painkiller that helps us to get effective relief from pain. This cannot cure pain. Yet when it is taken with proper rest, it can offer you effective relief from pain.
    This painkiller can offer you relief from any kind of pain. But Soma 350 mg is best in treating acute pain. Acute pain is a type of short-term pain which is sharp in nature. Buy Soma 350 mg online to get relief from your acute pain.

    https://globalonlinepills.com/product/soma-350-mg/


    Buy Soma 350 mg
    Soma Pill
    Buy Soma 350 mg online



    Buy Soma 350 mg online
    Soma Pill
    Buy Soma 350 mg

    ReplyDelete
  20. Wonderful blog post, thank you so much for the great information which you provided.
    https://rokucomlinks.org/

    ReplyDelete
  21. Hiiii....Thanks for sharing Great info...Nice post...Keep move on...
    Python Training in Hyderabad

    ReplyDelete
  22. Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. Thank you and good luck for the upcoming articles machine learning training

    ReplyDelete
  23. Thanks for sharing, very informative. Top Programming Languages in 2020.

    ReplyDelete
  24. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    angular js online training
    best angular js online training
    top angular js online training

    ReplyDelete
  25. Really a awesome blog for the freshers. Thanks for posting the information.
    Data Science Training in Delhi
    Data Science Course in Delhi

    ReplyDelete
  26. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
    data analytics courses in mumbai
    data science interview questions

    ReplyDelete
  27. I finally found great post here.I will get back here. I just added your blog to my bookmark sites. thanks.Quality posts is the crucial to invite the visitors to visit the web page, that's what this web page is providing.
    data science course Mumbai
    data science interview questions
    data analytics course in mumbai

    ReplyDelete
  28. This degree will be acceptable not only nationally as well as globally also. Digital marketing is the best source to make your carrier bright.
    ExcelR Digital Marketing Courses In Bangalore

    ReplyDelete
  29. Hello Admin!

    Thanks for the post. It was very interesting and meaningful. I really appreciate it! Keep updating stuffs like this. If you are looking for the Advertising Agency in Chennai | Printing in Chennai , Visit Inoventic Creative Agency Today..

    ReplyDelete
  30. Hey there! I simply wish to offer you a big thumbs up for the excellent information you have right here on this post. I am coming back to your web site for more soon.
    Techno

    ReplyDelete
  31. Thanks for sharing such a great information..Its really nice and informative.. software testing course

    ReplyDelete
  32. live draw hk banyak terdapat perbedaan dari situs2 penyedia live hk ini. Sedikit saran apabila anda setiap harinya mengikuti live draw hk kami pastikan selalu menggunakan situs live hk yang kami rekomendasikan saja. Termasuk juga untuk live draw sgp ataupun live sgp alternatif ini.

    ReplyDelete
  33. nice blogs
    great information.
    VLCC Institute Makeup Courses give you the platform to become a Professional Makeup Artist by learning from the best in the Beauty Industry.

    ReplyDelete
  34. This post is really nice and informative. The explanation given is really comprehensive and informative. I also want to say about the digital marketing course

    ReplyDelete
  35. Awesome article, it was exceptionally helpful! I simply began in this and I'm becoming more acquainted with it better. The post is written in very a good manner and it contains many useful information for me. Thank you very much and will look for more postings from you.


    digital marketing blog
    digital marketing bloggers
    digital marketing blogs
    digital marketing blogs in india
    digital marketing blog 2020
    digital marketing blog sites
    skartec's digital marketing blog
    skartec's blog
    digital marketing course
    digital marketing course in chennai
    digital marketing training
    skartec digital marketing academy

    ReplyDelete
  36. Good information and, keep sharing like this.

    7 tips to start a career in digital marketing

    “Digital marketing is the marketing of product or service using digital technologies, mainly on the Internet, but also including mobile phones, display advertising, and any other digital medium”. This is the definition that you would get when you search for the term “Digital marketing” in google. Let’s give out a simpler explanation by saying, “the form of marketing, using the internet and technologies like phones, computer etc”.

    we have offered to the advanced syllabus course digital marketing for available join now

    more details click the link now

    https://www.webdschool.com/digital-marketing-course-in-chennai.html

    ReplyDelete
  37. Amazing blog is very nice

    Web designing trends in 2020

    When we look into the trends, everything which is ruling today’s world was once a start up and slowly begun getting into. But Now they have literally transformed our lives on a tremendous note. To name a few, Facebook, Whats App, Twitter can be a promising proof for such a transformation and have a true impact on the digital world.

    we have offered to the advanced syllabus course web design and development for available join now

    more details click the link now

    https://www.webdschool.com/web-development-course-in-chennai.html

    ReplyDelete
  38. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. digital marketing course Bangalore

    ReplyDelete
  39. I have to search sites with relevant information on given topic ExcelR Digital Marketing Class In Pune and provide them to teacher our opinion and the article.

    ReplyDelete

  40. Nice to see this BLOG..keep updating for infromation We offers students hands-on opportunity to implement their knowledge gained in the real time projects & gain valuable experience that will make them a more eligible candidate for jobs.Digital Lync offers one of the best Full Stack training in Hyderabad with a comprehensive course curriculum with Continuous Integration, Delivery, and Testing.
    software training and placement institutes in hyderabad
    best training and placement institutes in hyderabad
    web development courses
    full stack developer course

    ReplyDelete
  41. This is a wonderful article, Given so much info in it, Thanks for sharing. CodeGnan offers courses in new technologies and makes sure students understand the flow of work from each and every perspective in a Real-Time environmen python training in vijayawada. , data scince training in vijayawada . , java training in vijayawada. ,

    ReplyDelete
  42. Impressive! I finally found great post here. Nice article on data science . It's really a nice experience to read your post. Thanks for sharing your innovative ideas to our vision.
    Data Science Course
    Data Science Course in Marathahalli
    Data Science Course Training in Bangalore

    ReplyDelete
  43. Thanks for the interesting blog that you have implemented here. Very helpful and innovative. Waiting for your next upcoming article.
    Java training in chennai
    Java training institute in chennai
    Java course in chennai
    Java training classes
    Java training
    Java programming classes
    core java coure
    <a

    ReplyDelete
  44. Thanks for this blog are more informative contents step by step. I here attached my site would you see this blog.

    7 tips to start a career in digital marketing

    “Digital marketing is the marketing of product or service using digital technologies, mainly on the Internet, but also including mobile phones, display advertising, and any other digital medium”. This is the definition that you would get when you search for the term “Digital marketing” in google. Let’s give out a simpler explanation by saying, “the form of marketing, using the internet and technologies like phones, computer etc”.

    we have offered to the advanced syllabus course digital marketing for available join now.

    more details click the link now.

    https://www.webdschool.com/digital-marketing-course-in-chennai.html

    ReplyDelete
  45. Amazing article useful information.

    Web designing trends in 2020

    When we look into the trends, everything which is ruling today’s world was once a start up and slowly begun getting into. But Now they have literally transformed our lives on a tremendous note. To name a few, Facebook, Whats App, Twitter can be a promising proof for such a transformation and have a true impact on the digital world.

    we have offered to the advanced syllabus course web design and development for available join now.

    more details click the link now.

    https://www.webdschool.com/web-development-course-in-chennai.html



    ReplyDelete
  46. Everything is very open with a really clear clarification of the issues. It was really informative. get Your website is very useful. Many thanks for sharing!

    ReplyDelete
  47. This is a topic that's near to my heart... Web develop Thank you! Exactly where are your contact details though?

    ReplyDelete
  48. I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business.satta king

    ReplyDelete
  49. I become greater than happy to uncover this notable internet site on line. I need to to thank you to your time due to this brilliant examine latest!! I genuinely loved each bit of it and I have you ever bookmarked to look new information to your blog.

    ReplyDelete
  50. Your blog is splendid, I follow and read continuously the blogs that you share, they have some really important information. M glad to be in touch plz keep up the good work.
    Digital Marketing Courses in Pune

    ReplyDelete
  51. I have to search sites with relevant information on given topic and provide them to teacher our opinion and the article.
    ExcelR digital marketing courses in mumbai

    ReplyDelete
  52. keep up the good work. this is an Ossam post. This is to helpful, i have read here all post. i am impressed. thank you. this is our courses in data analytics
    courses in data analytics | https://www.excelr.com/data-analytics-certification-training-course-in-mumbai

    ReplyDelete
  53. Really Very helpful Post & thanks for sharing & keep up the good work.
    Oflox Is The Best Digital Marketing Company In Dehradun Or Website Design Company In Dehradun

    ReplyDelete
  54. wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
    Data science Interview Questions
    Data Science Course

    ReplyDelete
  55. Really Nice Post, Thanks for sharing & keep up the good work.
    Oflox Is The Website Design Service In Dehradun

    ReplyDelete
  56. This is an awesome motivating article. I am practically satisfied with your great work. You put truly extremely supportive data. Keep it up. Continue blogging. Hoping to be perusing your next post
    Courses in Digital Marketing in Pune

    ReplyDelete
  57. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    workday studio online training
    best workday studio online training
    top workday studio online training

    ReplyDelete
  58. It is actually a great and helpful piece of information about Java. I am satisfied that you simply shared this helpful information with us. Please stay us informed like this. Thanks for sharing.

    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  59. Thanks for sharing this informatiions.
    artificial intelligence training in coimbatore

    Blue prism training in coimbatore

    RPA Course in coimbatore

    C and C++ training in coimbatore

    big data training in coimbatore

    hadoop training in coimbatore

    aws training in coimbatore

    ReplyDelete
  60. Thanks for the informative article About Java. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  61. Learn digital marketing live, Coimbatore – an institute perfected in coaching the digital marketing training course by covering the following topics said to be SEO, SEM, SMM and Email marketing that would educate individuals in the finest part of the choice.

    digital marketing course in coimbatore
    digital marketing training course in coimbatore
    digital marketing training in coimbatore

    ReplyDelete
  62. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data scientist course in hyderabad with placement

    ReplyDelete
  63. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data scientist course in hyderabad with placement

    ReplyDelete
  64. wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries. keep it up.
    data analytics course in Bangalore

    ReplyDelete
  65. First of all I would like to thank you for writing this post I love both writing and reading new posts and I was just looking at new posts to see me something new, only then I saw your post and the rest of the post is praiseworthy.
    sofeeya.com

    ReplyDelete
  66. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, best online data science courses

    ReplyDelete
  67. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, best online data science courses

    ReplyDelete
  68. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.....bangalore digital marketing course

    ReplyDelete
  69. I am very impressed by your excellent writing abilities. Please keep up the good informational writing.
    SAP training in Kolkata
    Best SAP training in Kolkata
    SAP training institute in Kolkata

    ReplyDelete
  70. I must say you are very skilled at persuasive writing if you can convince me to share your views
    SAP training in Mumbai
    Best SAP training in Mumbai
    SAP training institute Mumbai

    ReplyDelete
  71. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course

    ReplyDelete
  72. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course

    ReplyDelete
  73. wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
    Data Science Course

    ReplyDelete
  74. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science training in Hyderabad

    ReplyDelete
  75. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science training in Hyderabad

    ReplyDelete
  76. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course in hyderabad with placements

    ReplyDelete
  77. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course in hyderabad with placements

    ReplyDelete
  78. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, best online data science courses

    ReplyDelete
  79. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, best online data science courses

    ReplyDelete
  80. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science courses

    ReplyDelete
  81. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science courses

    ReplyDelete
  82. Great post i must say and thanks for the information. Education is definitely a sticky subject. However, is still among the leading topics of our time. I appreciate your post and look forward to more.

    Correlation vs Covariance

    ReplyDelete
  83. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science training

    ReplyDelete
  84. There are many things mentioned in this article I would not have thought of on my own. This material is inspirational, interesting and it allows the readers to open up their minds to original thinking.
    SAP training in Kolkata
    SAP training Kolkata
    Best SAP training in Kolkata
    SAP course in Kolkata
    SAP training institute Kolkata

    ReplyDelete
  85. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science certification

    ReplyDelete
  86. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science online course

    ReplyDelete
  87. Bài viết rất hay: Chúng tôi chuyên cung cấp các sản phẩm chất lượng



    Tác dụng thần kỳ của giảo cổ lam 7 lá



    Giảo cổ lam giá rẻ tại Hà Nội



    Bao nhiêu tiền 1 kg giảo cổ lam


    ReplyDelete
  88. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data sciecne course in hyderabad

    ReplyDelete
  89. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Correlation vs Covariance

    ReplyDelete
  90. Very Interesting blog. Thanks for sharing this blog. You can also go through blog which is related to this which is providing variety of movies also. For details you can visit our official website.

    ReplyDelete
  91. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course

    ReplyDelete
  92. This Was An Amazing ! I Haven't Seen This Type of Blog Ever ! Thankyou For Sharing, data science course

    ReplyDelete
  93. Bài viết rất hay: Chúng tôi chuyên cung cấp các sản phẩm chất lượng sau:

    Lều xông hơi tại nhà



    Lều xông hơi hồng ngoại



    Cảm ơn các bạn!

    ReplyDelete
  94. A backlink is a link created when one website links to another. Backlinks are important to SEO & impact for higher ranking. In my 7+ years seo Career i see, without backlinks a website doesn't rank higher on google SERP.

    Get Your 300+ High Quality DoFollow Backlinks Here!

    Order Now with Full Confidence & 100% satisfaction.

    ReplyDelete
  95. Great post! I am actually getting ready to across this information, It’s very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well. share more.
    Ai & Artificial Intelligence Course in Chennai
    PHP Training in Chennai
    Ethical Hacking Course in Chennai Blue Prism Training in Chennai
    UiPath Training in Chennai

    ReplyDelete