Python/maths/relu.py

41 lines
1.1 KiB
Python
Raw Normal View History

2020-03-13 07:33:36 +00:00
"""
This script demonstrates the implementation of the ReLU function.
It's a kind of activation function defined as the positive part of its argument in the
context of neural network.
2020-03-13 07:33:36 +00:00
The function takes a vector of K real numbers as input and then argmax(x, 0).
After through ReLU, the element of the vector always 0 or real number.
Script inspired from its corresponding Wikipedia article
https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
"""
from __future__ import annotations
2020-03-13 07:33:36 +00:00
import numpy as np
def relu(vector: list[float]):
2020-03-13 07:33:36 +00:00
"""
Implements the relu function
2020-03-13 07:33:36 +00:00
Parameters:
vector (np.array,list,tuple): A numpy array of shape (1,n)
consisting of real values or a similar list,tuple
2020-03-13 07:33:36 +00:00
Returns:
relu_vec (np.array): The input numpy array, after applying
relu.
2020-03-13 07:33:36 +00:00
>>> vec = np.array([-1, 0, 5])
>>> relu(vec)
array([0, 0, 5])
2020-03-13 07:33:36 +00:00
"""
# compare two arrays and then return element-wise maxima.
return np.maximum(0, vector)
if __name__ == "__main__":
print(np.array(relu([-1, 0, 5]))) # --> [0, 0, 5]