Remove nan from 2D NumPy ndarray

Numpy

In this post, we are going to understand how to Remove nan from 2D NumPy ndarray. To delete rows and columns with NAN values first we need to find the nan values in numpy ndarray using isnan() function.

1. Remove all missing values from NumPy ndarray

By using np. isnan() function, we will get ndarray that has True at the place of missing value and False at other values. We are using the negation operator (~) on the ndarray, it returns the opposite of np.isnan() function return False at the place of missing value and True at the place of other value.,

By using the negation operator (~) on the ndarray, we will get a ndarray that contains no missing values this is how to remove all nan values from NumpY ndarray

import numpy as np
  
  
myarr = np.array([[12,14,np.nan,80],
                  [np.nan,75,60,np.nan],
                  [3,6,np.nan,12],
                  [np.nan,8,12,16]])

print('nan values in array :\n',np.isnan(myarr))

#removing all missing values from nd array
newarr = myarr[~np.isnan(myarr)]

print('\n array after removing all missing values :\n',newarr)

Output

nan value in array :
 [[False False  True False]
 [ True False False  True]
 [False False  True False]
 [ True False False False]]

 array after removing all missing values :
 [12. 14. 80. 75. 60.  3.  6. 12.  8. 12. 16.]

2. Remove rows with nan values from NumPy ndarray

We are using the any() method to get the row that contains at least one nan value. The axis=1 helps us to look for ndarray that has at least one missing value(true) for each row.

The negation operator (~) on the ndarray omits the rows with nan/missing values and returns the numpy array after removing the rows with missing values (nan).

import numpy as np
  
  
myarr = np.array([[12,70,80,25],
                  [25,np.nan,50,20],
                  [3,np.nan,9,12],
                  [4,8,9,16]])



print('nan value in array :\n',np.isnan(myarr).any(axis=1))


newarr = myarr[~np.isnan(myarr).any(axis=1)]



print('\n array after removing rows with missing values :\n',newarr)

Output

nan value in array rows :
 [False  True  True False]

 array after removing rows with missing values :
 [[12. 70. 80. 25.]
 [ 4.  8.  9. 16.]]

3. Remove rows with all elements nan values

To remove rows all elements have nan values, We can use python all() function along with axis =1. It will remove rows that have all elements has missing values.

import numpy as np
  
  
myarr = np.array([[12,70,80,25],
                  [np.nan,np.nan,np.nan,np.nan],
                  [3,np.nan,9,12],
                  [4,8,9,16]])



print('nan value in array :\n',np.isnan(myarr).all(axis=1))


newarr = myarr[~np.isnan(myarr).all(axis=1)]



print('\n array after removing rows with missing values :\n',newarr)

Output

rows with nan value in array :
 [False  True  True False]

 array after removing rows with missing values :
 [[12. 70. 80. 25.]
 [ 3. nan  9. 12.]
 [ 4.  8.  9. 16.]]

4. Remove columns with nan values

We are using any() method along with axis=0 to get the ndarray columns that contain at least one nan value. ndarray that has at least one missing value(true) for each column.

The negation operator (~) on the ndarray returns the numpy array after removing the columns with missing values (nan).

import numpy as np
  
myarr = np.array([[12,70,80,25],
                  [21,np.nan,50,20],
                  [3,np.nan,9,12],
                  [4,np.nan,9,16]])




print('nan value in array :\n',np.isnan(myarr).any(axis=0))


print(myarr[:,~np.isnan(myarr).any(axis=0)])

Output

nan value in array :
 [False  True False False]
[[12. 80. 25.]
 [21. 50. 20.]
 [ 3.  9. 12.]
 [ 4.  9. 16.]]

5. Remove columns with all nan values

To remove columns all elements have nan values, We can use python all() function along with axis =1. It will remove rows that have all elements has missing values. 

import numpy as np
  
myarr = np.array([[12,np.nan,80,25],
                  [21,np.nan,50,20],
                  [3,np.nan,9,12],
                  [4,np.nan,9,16]])




print('nan value in array :\n',np.isnan(myarr).any(axis=0))


print(myarr[:,~np.isnan(myarr).all(axis=0)])

Output

nan value in array :
 [False  True False False]
[[12. 80. 25.]
 [21. 50. 20.]
 [ 3.  9. 12.]
 [ 4.  9. 16.]]

Summary

In this post we have learned how to Remove nan from 2D NumPy ndarray

Related Posts