mathdatech/Python
1
0
Fork 0
mirror of https://github.com/TheAlgorithms/Python.git synced 2024-11-23 21:11:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

The ELU activation is added (#8699)

Browse Source 
* tanh function been added

* tanh function been added

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tanh function is added

* tanh function is added

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* tanh function added

* tanh function added

* tanh function is added

* Apply suggestions from code review

* ELU activation function is added

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* elu activation is added

* ELU activation is added

* Update maths/elu_activation.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Exponential_linear_unit activation is added

* Exponential_linear_unit activation is added

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Christian Clauss <cclauss@me.com>
This commit is contained in:
Dipankar Mitra 2023-05-02 20:06:28 +05:30 committed by GitHub
parent 777f966893
commit 7310514509
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 40 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

40
neural_network/activation_functions/exponential_linear_unit.py Normal file
View File

@ -0,0 +1,40 @@
"""
Implements the Exponential Linear Unit or ELU function.
The function takes a vector of K real numbers and a real number alpha as
input and then applies the ELU function to each element of the vector.
Script inspired from its corresponding Wikipedia article
https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
"""
import numpy as np
def exponential_linear_unit(vector: np.ndarray, alpha: float) -> np.ndarray:
"""
Implements the ELU activation function.
Parameters:
vector: the array containing input of elu activation
alpha: hyper-parameter
return:
elu (np.array): The input numpy array after applying elu.
Mathematically, f(x) = x, x>0 else (alpha * (e^x -1)), x<=0, alpha >=0
Examples:
>>> exponential_linear_unit(vector=np.array([2.3,0.6,-2,-3.8]), alpha=0.3)
array([ 2.3 , 0.6 , -0.25939942, -0.29328877])
>>> exponential_linear_unit(vector=np.array([-9.2,-0.3,0.45,-4.56]), alpha=0.067)
array([-0.06699323, -0.01736518, 0.45 , -0.06629904])
"""
return np.where(vector > 0, vector, (alpha * (np.exp(vector) - 1)))
if __name__ == "__main__":
import doctest
doctest.testmod()