Explore all the technology expertise we have to develop AI solutions

Explore All

AI Insights. Dive into research, 

expert videos, and our
R&D solutions

Get to know us, our leadership, development direction, and why we call ourselves applied AI experts.

Look at our open positions and join the applied AI revolution!

Open Positions
Home Resources Applying deep learning to right whale photo identification

Applying deep learning to right whale photo identification

7 minutes

read

•

Read full paper

Open PDF

Details

Joint paper with National Oceanic and Atmospheric Administration, University of Warsaw

  • The research paper presents the winning solution of a computer vision competition organized by the NOAA Fisheries on the Kaggle.com data science platform.
  • The solution automatically identifies individual whales with an 87% accuracy using a series of convolutional neural networks to identify the region of interest on an image, then rotate, crop, and create standardized photographs of uniform size and orientation, and then identify the correct individual whale from these passport-like photographs.

Authors: Robert Bogucki, Marek Cygan, Christin Brangwynne Khan, Maciej Klimek, Jan Kanty Milczek, Marcin Mucha

Abstract

Photo identification is an important tool for estimating abundance and monitoring population trends over time. However, manually matching photographs to known individuals is time-consuming. Motivated by recent developments in image recognition, we hosted a data science challenge on the crowdsourcing platform Kaggle to automate the identification of endangered North Atlantic right whales (Eubalaena glacialis). The winning solution automatically identified individual whales with 87% accuracy with a series of convolutional neural networks to identify the region of interest on an image, rotate, crop, and create standardized photographs of uniform size and orientation and then identify the correct individual whale from these passport like photographs. Recent advances in deep learning coupled with this fully automated workflow have yielded impressive results and have the potential to revolutionize traditional methods for the collection of data on the abundance and distribution of wild populations. Presenting these results to a broad audience should further bridge the gap between the data science and conservation science communities.

References

  1. Agler BA, Beard JA, Bowman RS, Corbett HD, Frohock SE, Hawvermale MP, Katona SK, Sadove SS, Seipt IE. 1990. Fin whale (Balaenoptera physalus) photographic identification: methodology and prelimi-nary results from the western North Atlantic. Report of the Meeting of the International Whaling Commission 12:349–356.
  2. Aizenberg I, Aizenberg N, Vandewalle J. 2000. Multi-valued and uni- versal binary neurons: theory, learning and applications. Springer Science & Business Media, Germany.
  3. Arzoumanian Z, Holmberg J, Norman B. 2005. An astronomical pattern- matching algorithm for computer-aided identification of whale sharks Rhincodon typus. Journal of Applied Ecology 42:999–1011. Conservation Biology Volume 33, No. 3, 2019 15231739, 2019,
  4. Beijbom O, et al. 2015. Towards automated annotation of ben-thic survey images: variability of human experts and operational modes of automation. PLOS ONE 10 (e0130312) https://doi.org/10.1371/journal.pone.0130312.
  5. Bogucki R, Cygan M, Klimek M, Milczek JK, Mucha M. 2016. Which whale is it, anyway? Face recognition for right whales using deep learning. deepsense.ai, Palo Alto, California. Available from https://deepsense.ai/deep-learning-right-whale-recognition-kaggle/.
  6. Ciresan DC, Meier U, Gambardella LM, Schmidhuber J. 2010. Deep, big, simple neural nets for handwritten digit recognition. Neural Computation 22:3207–3220.
  7. Clapham PJ, Pace III RM. 2001. Defining triggers for temporary area closures to protect right whales from entanglements: issues and options. Northeast Fisheries Science Center Reference Document 01–06. U.S. Department of Commerce, Washington, D.C.
  8. Cole TVN, Gerrior P, Merrick RL. 2007. Methodologies and preliminary results of the NOAA National Marine Fisheries Service aerial survey program for right whales (Eubalaena glacialis) in the northeast U.S., 1998–2006. Northeast Fisheries Science Center Reference Document 07-02; U.S. Department of Commerce, Washington, D.C.
  9. Crall JP, Stewart CV, Berger-Wolf TY, Rubenstein DI, Sundaresan SR. HotSpotter – patterned species instance recognition. Proceedings of the IEEE workshop on applications of computer vision. Institute of Electrical and Electronics Engineers, Piscataway, New
    Jersey.
  10. Eberhardt LL. 1969. Population estimates from recapture frequencies. 33:28–39.
  11. Flukebook. 2018. Flukebook helps you study identify, and protect cetacean populations. Flukebook, Portland, Oregon. Available from http://www.flukebook.org/ (accessed November 2018).
  12. Goodfellow I, Bengio Y, Courville A. 2016. Deep learning. MIT Press, Cambridge, Massachusetts.
  13. Hamilton PK, Martin SM. 1999. A catalog of identified right whales from the Western North Atlantic: 1935 to 1997. New England Aquarium, Boston.
  14. He K, Zhang X, Ren S, Sun J. 2016. Deep residual learning for image recognition. The IEEE conference on computer vision and pattern recognition (CVPR) 2016:770–778.
  15. Henry AG, Cole TVN, Hall L, Ledwell W, Morin D, Reid A. 2016. Serious injury and mortality determinations for baleen whale stocks along the Gulf of Mexico, United States East Coast and Atlantic Canadian Provinces, 2010–2014. Northeast Fisheries Science Center reference document 16-10. U.S. Department of Commerce, Washington, D.C.
  16. Hiby L, Lovell P. 2001. A note on an automated system for matching the callosity patterns on aerial photographs of southern right whales. Journal of Cetacean Research and Management 2:291–295.
  17. Katona S, Whitehead H. 1981. Identifying humpback whales using their natural markings. Polar Record 20:439–444.
  18. Khan C, Duley P, Henry A, Gatzke J, Crowe L, Cole T. 2016. Nort Atlantic Right Whale Sighting Survey (NARWSS) and Right Whale Sighting Advisory System (RWSAS) 2014 results summary. Northeast Fisheries Science Center Reference Document 16-01. U.S. Department of Commerce, Washington, D.C.
  19. Kniest E. 2010. Fluke Matcher: a computer-aided matching system for humpback whale (Megaptera novaeangliae) flukes. Marine Mammal Science 26:744–756.
  20. Kraus SD, Moore KE, Price CA, Crone MJ, Watkins WA, Winn HE, Prescott JH. 1986. The use of photographs to identify individual North Atlantic right whales (Eubalaena glacialis). Report of the International Whaling Commission 10:145–151.
  21. Krizhevsky A, Sutskever I, Hinton G. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems (NIPS) 1:1097–1105.
  22. Long J, Shelhamer E, Darrell T. 2015. Fully convolutional networks for semantic segmentation. The IEEE Conference on Computer Vision and Pattern Recognition, Boston, Massachusetts, pp. 3431–3440. New England Aquarium. 2017. The north Atlantic right whale catalog. New England Aquarium, Boston. Available from http://rwcatalog.neaq.org (accessed March 2017).
  23. Oh K, Jung K. 2004. GPU implementation of neural networks. Pattern
    Recognition 37:1311–1314.
  24. Otis DL, Burnham KP, White GC, Anderson DR. 1978. Statistical inference from capture data on closed animal populations. Wildlife Monographs 62:3–135.
  25. Pace III RM, Corkeron PJ, Kraus SD. 2017. State–space markrecapture estimates reveal a recent decline in abundance of North Atlantic right whales. Ecology and Evolution 7:8730–8741.
  26. Payne R. 1976. At home with right whales. National Geographic Magazine 149:322–339.
  27. Raina R, Madhavan A, Ng AY. 2009. Large-scale deep unsupervised learning using graphics processors. Proceedings of the 26th International Conference on machine learning, Montreal, Canada.
  28. Redmon J, Divvala S, Girshick R, Farhadi A. 2016. You only look once: unified, real-time object detection. The IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788.
  29. Ren S, He K, Girshick R, Jian S. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, pp. 91–99.
  30. Russakovsky O, et al. 2015. ImageNet large scale visual recognition challenge. International Journal of Computer Vision 115:211–252.
  31. Russell SJ, Norvig P. 1995. Artificial intelligence: a modern approach. Prentice Hall, Upper Saddle River, New Jersey.
  32. Seber GA. 1982. The estimation of animal abundance and related parameters. Charles Griffin, London.
  33. Silber GK, Adams JD, Asaro MJ, Cole TVN, Moore KS, Ward-Geiger LI, Zoodsma BJ. 2015. The right whale mandatory ship reporting system: a retrospective. PeerJ 3:e866.
  34. Silber GK, Adams JD, Fonnesbeck CJ. 2014. Compliance with vessel speed restrictions to protect North Atlantic right whales. PeerJ 2:e399.
  35. Silver D, et al. 2016. Mastering the game of go with deep neural networks and tree search. Nature 529:484–489.
  36. Taigman Y, Yang M, Ranzato MA, Wolf L. 2014. DeepFace: closing the gap to human-level performance in face verification. The IEEE Con- ference on Computer Vision and Pattern Recognition 2014:1701–1708.
  37. Van der Hoop JM, Moore MJ, Barco SG, Cole TV, Daoust P, Henry AG, McAlpine DF, McLellan WA, Wimmer T, Solow AR. 2013. Assessment of management to mitigate anthropogenic effects on large whales. Conservation Biology 27:121–133.
  38. Vanderlaan ASM, Taggart CT, Serdynska AR, Kenney RD, Brown MW. Reducing the risk of lethal encounters: vessels and right whales in the Bay of Fundy and on the Scotian Shelf. Endangered Species Research 4:283–297.
  39. Waring GT, Josephson E, Maze-Foley K, Rosel PE, editors. 2016. US Atlantic and Gulf of Mexico marine mammal stock assessments-National Oceanic and Atmospheric Administration (NOAA) technical memorandum NMFS NE 238. NOAA.
  40. Wiley D, Hatch L, Thompson M, Schwehr K, MacDonald C. Marine sanctuaries and marine planning: protecting endangered marine life. Proceedings of the Marine Safety & Security Council, the Coast Guard Journal of Safety at Sea. Fall 2013: 10–15.
  41. Wursig B, Jefferson RA. 1990. Methods of photo-identification for small cetaceans. Report of the International Whaling Commission 12: 42–43.
  42. Xing F, Xie Y, Su H, Liu F, Yang L. 2017. Deep learning in microscopy image analysis: a survey. IEEE Transactions on Neural Networks and Learning Systems 99:1–19.

Share this post

Posted in

Explore more insights and resources

Transform your business
with AI solutions

Let’s talk