Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Neo: Hierarchical Confusion Matrix Visualization (CHI 2022)

License

NotificationsYou must be signed in to change notification settings

apple/ml-hierarchical-confusion-matrix

Repository files navigation

npm version

The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. Neo is a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications.

This code accompanies the research paper:

Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Jochen Görtler, Fred Hohman, Dominik Moritz, Kanit Wongsuphasawat, Donghao Ren, Rahul Nair, Marc Kirchner, Kayur Patel
ACM Conference on Human Factors in Computing Systems (CHI), 2022.
image Paper,image Live demo,image Video,image Video Preview,image Code

Documentation

You can embed our confusion matrix visualization into your own project. There are two ways to use it.

NPM

Install withnpm install --save @apple/hierarchical-confusion-matrix oryarn add @apple/hierarchical-confusion-matrix.

Then you can import the module in your project

importconfMatfrom'@apple/hierarchical-confusion-matrix';constspec={classes:['root'],};constconfusions=[{actual:['root:a'],observed:['root:a'],count:1,},{actual:['root:a'],observed:['root:b'],count:2,},{actual:['root:b'],observed:['root:a'],count:3,},{actual:['root:b'],observed:['root:b'],count:4,},];confMat.embed('matContainer',spec,confusions);

Embed the Compiled File

If you prefer to load the compiled JavaScript directly, you have to compile it. To do this, runyarn install and copy thepublic/confMat.js into your project. Here is a simple example of a small confusion matrix:

<!DOCTYPE html><html><head><metacharset="utf8"/><metaname="viewport"content="width=device-width"/><title>Neo: Hierarchical Confusion Matrix</title></head><body><divid="matContainer"></div><scriptsrc="confMat.js"></script><script>constspec={classes:['root'],};constconfusions=[{actual:['root:a'],observed:['root:a'],count:1,},{actual:['root:a'],observed:['root:b'],count:2,},{actual:['root:b'],observed:['root:a'],count:3,},{actual:['root:b'],observed:['root:b'],count:4,},];confMat.embed('matContainer',spec,confusions);</script></body></html>

Specification

You can find all the options that you can pass via thespec argument insrc/specification.ts.

Loaders

The different loaders can be found insrc/loaders, which include loading data fromjson,csv,vega, and a synthetic examplesynth for testing.

Confusion Data Format Examples

Example 1: Conventional Confusions

The confusions for data withactual labels offruit:lemon that are incorrectly predicted asfruit:apple, of which there arecount 1 of them.

{"actual": ["fruit:lemon"],"observed": ["fruit:apple"],"count":1}

Example 2: Hierarchical Confusions

The confusions for hierarchical data withactual labels offruit:citrus:lemon that are incorrectly predicted asfruit:pome:apple, of which there arecount 2 of them. Note: denotes hierarchies.

{"actual": ["fruit:citrus:lemon"],"observed": ["fruit:pome:apple"],"count":2}

Example 3: Multi-output Confusions

The confusions for multi-output data withactual labels offruit:lemon,taste:sweet that are incorrectly predicted asfruit:apple,taste:sour, of which there arecount 3 of them. Note, denotes multi-ouput labels.

{"actual": ["fruit:lemon","taste:sweet"],"observed": ["fruit:apple","taste:sour"],"count":3}

Example 4: Hierarchical and Multi-output Confusions

The confusions for hierarchical and multi-output data withactual labels offruit:citrus:lemon,taste:sweet,ripeness:ripe that are incorrectly predicted asfruit:pome:apple,taste:sour,ripeness:not-ripe, of which there arecount 4 of them.

{"actual": ["fruit:citrus:lemon","taste:sweet","ripeness:ripe"    ],"observed": ["fruit:pome:apple","taste:sour""ripeness:not-ripe"    ],"count":4}

Seefruit.json for a complete example of confusions for a hierarchical fruit, taste, and ripeness classification model.

Development

Build:

yarn installyarn build

Test:

yarn test:unit

Dev Server:

yarn dev

Lint & Fix:

yarn lint

Contributing

When making contributions, refer to theCONTRIBUTING guidelines and read theCODE OF CONDUCT.

BibTeX

To cite our paper, please use:

@inproceedings{goertler2022neo,title={Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels},author={Görtler, Jochen and Hohman, Fred and Moritz, Dominik and Wongsuphasawat, Kanit and Ren, Donghao and Nair, Rahul and Kirchner, Marc and Patel, Kayur},booktitle={Proceedings of the SIGCHI Conference on Human Factors in Computing Systems},year={2022},organization={ACM},doi={10.1145/3491102.3501823}}

License

This code is released under theLICENSE terms.

Contributors4

  •  
  •  
  •  
  •  

Languages

[8]ページ先頭
©2009-2025 Movatter.jp