Home > Research > Publications & Outputs > Explaining Deep Learning Models Through Rule-Ba...

Electronic data

  • Explaining Deep Learning Models Through Rule-Based Approximation and Visualization

    Rights statement: ©2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

    Accepted author manuscript, 834 KB, PDF document

    Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License

Links

Text available via DOI:

View graph of relations

Explaining Deep Learning Models Through Rule-Based Approximation and Visualization

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Explaining Deep Learning Models Through Rule-Based Approximation and Visualization. / Almeida Soares, Eduardo; Angelov, Plamen; Costa, Bruno; Castro, Marcos; Nageshrao, Subramanya ; Filev, Dimitar.

In: IEEE Transactions on Fuzzy Systems, Vol. 29, No. 8, 31.08.2021, p. 2399-2407.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

Almeida Soares, E, Angelov, P, Costa, B, Castro, M, Nageshrao, S & Filev, D 2021, 'Explaining Deep Learning Models Through Rule-Based Approximation and Visualization', IEEE Transactions on Fuzzy Systems, vol. 29, no. 8, pp. 2399-2407. https://doi.org/10.1109/TFUZZ.2020.2999776

APA

Vancouver

Author

Almeida Soares, Eduardo ; Angelov, Plamen ; Costa, Bruno ; Castro, Marcos ; Nageshrao, Subramanya ; Filev, Dimitar. / Explaining Deep Learning Models Through Rule-Based Approximation and Visualization. In: IEEE Transactions on Fuzzy Systems. 2021 ; Vol. 29, No. 8. pp. 2399-2407.

Bibtex

@article{14e08731943644c39fb388d09da1598e,
title = "Explaining Deep Learning Models Through Rule-Based Approximation and Visualization",
abstract = "This paper describes a novel approach to the problem of developing explainable machine learning models. We consider a Deep Reinforcement Learning (DRL) model representing a highway path planning policy for autonomous highway driving. The model constitutes a mapping from the continuous multidimensional state space characterizing vehicle positions and velocities to a discrete set of actions in longitudinal and lateral direction. It is obtained by applying a customized version of the Double Deep Q-Network (DDQN) learning algorithm. The main idea is to approximate the DRL model with a set of IF…THEN rules that provide an alternative interpretable model, which is further enhanced by visualizing the rules. This concept is rationalized by the universal approximation properties of the rule-based models with fuzzy predicates. The proposed approach includes a learning engine composed of 0-order fuzzy rules, which generalize locally around the prototypes by using multivariate function models. The adjacent (in the data space) prototypes, which correspond to the same action are further grouped and merged into so-called {"}MegaClouds{"} reducing significantly the number of fuzzy rules. The input selection method is based on ranking the density of the individual inputs. Experimental results show that the specific DRL agent can be interpreted by approximating with families of rules of different granularity. The method is computationally efficient and can be potentially extended to addressing the explainability of the broader set of fully connected deep neural network models",
keywords = "Deep Reinforcement Learning, explainable AI, rule-based models, prototype- and density-based models, density-based input selection, autonomous driving",
author = "{Almeida Soares}, Eduardo and Plamen Angelov and Bruno Costa and Marcos Castro and Subramanya Nageshrao and Dimitar Filev",
note = "{\textcopyright}2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. ",
year = "2021",
month = aug,
day = "31",
doi = "10.1109/TFUZZ.2020.2999776",
language = "English",
volume = "29",
pages = "2399--2407",
journal = "IEEE Transactions on Fuzzy Systems",
issn = "1063-6706",
publisher = "IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC",
number = "8",

}

RIS

TY - JOUR

T1 - Explaining Deep Learning Models Through Rule-Based Approximation and Visualization

AU - Almeida Soares, Eduardo

AU - Angelov, Plamen

AU - Costa, Bruno

AU - Castro, Marcos

AU - Nageshrao, Subramanya

AU - Filev, Dimitar

N1 - ©2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

PY - 2021/8/31

Y1 - 2021/8/31

N2 - This paper describes a novel approach to the problem of developing explainable machine learning models. We consider a Deep Reinforcement Learning (DRL) model representing a highway path planning policy for autonomous highway driving. The model constitutes a mapping from the continuous multidimensional state space characterizing vehicle positions and velocities to a discrete set of actions in longitudinal and lateral direction. It is obtained by applying a customized version of the Double Deep Q-Network (DDQN) learning algorithm. The main idea is to approximate the DRL model with a set of IF…THEN rules that provide an alternative interpretable model, which is further enhanced by visualizing the rules. This concept is rationalized by the universal approximation properties of the rule-based models with fuzzy predicates. The proposed approach includes a learning engine composed of 0-order fuzzy rules, which generalize locally around the prototypes by using multivariate function models. The adjacent (in the data space) prototypes, which correspond to the same action are further grouped and merged into so-called "MegaClouds" reducing significantly the number of fuzzy rules. The input selection method is based on ranking the density of the individual inputs. Experimental results show that the specific DRL agent can be interpreted by approximating with families of rules of different granularity. The method is computationally efficient and can be potentially extended to addressing the explainability of the broader set of fully connected deep neural network models

AB - This paper describes a novel approach to the problem of developing explainable machine learning models. We consider a Deep Reinforcement Learning (DRL) model representing a highway path planning policy for autonomous highway driving. The model constitutes a mapping from the continuous multidimensional state space characterizing vehicle positions and velocities to a discrete set of actions in longitudinal and lateral direction. It is obtained by applying a customized version of the Double Deep Q-Network (DDQN) learning algorithm. The main idea is to approximate the DRL model with a set of IF…THEN rules that provide an alternative interpretable model, which is further enhanced by visualizing the rules. This concept is rationalized by the universal approximation properties of the rule-based models with fuzzy predicates. The proposed approach includes a learning engine composed of 0-order fuzzy rules, which generalize locally around the prototypes by using multivariate function models. The adjacent (in the data space) prototypes, which correspond to the same action are further grouped and merged into so-called "MegaClouds" reducing significantly the number of fuzzy rules. The input selection method is based on ranking the density of the individual inputs. Experimental results show that the specific DRL agent can be interpreted by approximating with families of rules of different granularity. The method is computationally efficient and can be potentially extended to addressing the explainability of the broader set of fully connected deep neural network models

KW - Deep Reinforcement Learning

KW - explainable AI

KW - rule-based models

KW - prototype- and density-based models

KW - density-based input selection

KW - autonomous driving

U2 - 10.1109/TFUZZ.2020.2999776

DO - 10.1109/TFUZZ.2020.2999776

M3 - Journal article

VL - 29

SP - 2399

EP - 2407

JO - IEEE Transactions on Fuzzy Systems

JF - IEEE Transactions on Fuzzy Systems

SN - 1063-6706

IS - 8

ER -