Home > Mateda2.0 > knowledge_extraction > entropy.m

# entropy

## PURPOSE

function H = entropy(var,min,max,laplace)

## SYNOPSIS

function H = entropy(var,min,max,laplace)

## DESCRIPTION

     function H = entropy(var,min,max,laplace)

     entropy: Calculates the entropy for a discrete variables with positive states from min to max.
     INPUT:
     min: The minimum value
     max: The maximum value
     var: Array with the values for the discrete variable
     laplace: Determines wether or not Laplace correction is used in the computation
     of the probabilities. Laplace=1 (It is used), otherwise  it is not.
     OUTPUT:
     H: The entropy for a given variable.

     Last version 5/11/08. Carlos Echegoyen (carlos.echegoyen@ehu.es)

## CROSS-REFERENCE INFORMATION

This function calls:

This function is called by:

- Individuals_entropy

## SOURCE CODE

    0001 function H = entropy(var,min,max,laplace)
    0002 % function H = entropy(var,min,max,laplace)
    0003 %
    0004 % entropy: Calculates the entropy for a discrete variables with positive states from min to max.
    0005 % INPUT:
    0006 % min: The minimum value
    0007 % max: The maximum value
    0008 % var: Array with the values for the discrete variable
    0009 % laplace: Determines wether or not Laplace correction is used in the computation
    0010 % of the probabilities. Laplace=1 (It is used), otherwise  it is not.
    0011 % OUTPUT:
    0012 % H: The entropy for a given variable.
    0013 %
    0014 % Last version 5/11/08. Carlos Echegoyen (carlos.echegoyen@ehu.es)
    0015
    0016
    0017 if size(var,2) == 1
    0018     var = var';
    0019 end
    0020
    0021 H = 0;
    0022 N = size(var,2);
    0023
    0024 % One counter for each state
    0025 nstates = max-min+1;
    0026 cont=zeros(1,nstates);
    0027 c=0;
    0028
    0029 for j=min:max % sumo 1 por si hay ceros. Recorro los estados
    0030     c = c + 1;
    0031     cont(c) = sum(var==j);
    0032
    0033     % Calculating probabilities
    0034     if (laplace==1)
    0035      Probs(c) = (cont(c)+1) / (N+nstates);
    0036     else
    0037       Probs(c) = (cont(c)) / (nstates);
    0038     end
    0039
    0040     % Calculating entropy
    0041     if Probs(c)>0
    0042       H = H + Probs(c) * log2(Probs(c));
    0043     end,
    0044 end
    0045
    0046 H = -H;
    0047
    0048