NumPy in DataScience

,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

It is a predefined library of python which is used to perform a mathematical operation using predefined methods.
NumPy uses array-type data to perform the operation.
it contains array-type data to accept input data.
Operation using NumPy:-
Using NumPy, a developer can perform the following operations −
⦁    Mathematical and logical operations on arrays.
⦁    Fourier transforms and routines for shape manipulation.
⦁    Operations related to linear algebra. NumPy has in-built functions for linear algebra and random number generation.

What is the Difference between NumPy Array and List?

1)  Numpy array has n-th dimensional means it can be one dimension to nth dimension but the list has only 1 or 2 dimension approach.

2) Numpy array is used to contain data of data science to implement multiple functionalities of scipy, NumPy, pandas, and matplotlib but the list is used to provide python core functionality.

3) Numpy array speed and performance are best as compare to List.

4)  Numpy array space is fixed but the list is resizable.

5)  Numpy array can contain datatype and dimension attribute to store similar type of elements using proper sequence.

Q) Can we convert the list objects to NumPy array?

Answer)  Yes, we can convert the NumPy array to a list.

import numpy as np

arr = [10,20,30,70,11]

a = np.array(arr)

print(a)

How to install NumPy:-

before installation of NumPy python, pip, and python environment paths should be set.
open Command Prompt and Type:-
pip install NumPy

First, you should install the Jupyter notebook offline version or online version.

https://jupyter.org/try

Numpy array:-
It is called NDArray, which means we can create one-dimension,two-dimension, and multidimensional arrays using Numpy.
import numpy as np
a = np.array([[1, 2], [3, 4]])
print(a)
Display elements on NumPy Multidimensional Array?

a = np.array(([1,2],[3,4]))

for i in range(0,2):

    for j in range(0,2):

        print(a[i][j],end=' ')

    print()  

Assignments:-

1)  WAP to calculate the sum and multiply two different matrices using NumPy array?

2)  WAP to calculate Sum of row elements into NumPy array?

3)   WAP to display total prime elements into NumPy array?

Solve these assignments of NumPy array?

WAP to find max element in NumPy array?

WAP to sort the elements of the NumPy array?

WAP to split one array into two different subarrays?

WAP to merge two NumPy arrays into one array?

Solution of this program?

import numpy as np

a = np.array([1,2,3])

b = np.array([4,5,6])

size= len(a)+len(b)

c = np.array([int]*size)

for i in range(0,size):

    if i<len(a):

        c[i]=a[i]

    else:

        c[i]=b[i-len(a)]       

        

print(c)       

WAP to display prime elements in Numpy array

How many ways to declare NumPy array:-

1)  using NumPy array():-

    array() is used to convert list objects to NumPy array

    It is used to create a NumPy array using common type elements and specific elements both.     

     arrayname =  numpy.array([elements,....],dtyle='datatype'])

    Example:-

    import numpy as np

    a = np.array([1,2,3,4,5])
    print(a)    

2)  using numpy arange():-  

    It is used to  create a NumPy array using starting and ending points with increment value

    arrayname = numpy.arange(start,end,increment)    

  Example

   import numpy as np

  arr = np.arange(1,10,2,dtype='float')

  print(arr)

3)  using linspace():-

It is used to display elements based on the range with the same distance interval, distance will be calculated by starting value and ending value.

arrayname = numpy.linspace(start,end,increment)

import numpy as np

arr = np.linspace(1,10,5)

arr

4)  using empty():-

It is used to display random elements, we will provide a number of rows and columns using empty().

It will create random elements.

arrayname = numpy.empty((row,column),dtype='datatype')

import numpy as np

arr = np.empty((3,3),dtype='int')

arr

Numpy Operation:-
Numpy provides various predefined methods to manage array operation
1) ndmin:-  it is used to convert array to multiple dimension
a = np.array([1,2,3,4],ndmin=2)
2) dtype:-  It is used to convert array elements to different types of elements.
a = np.array([1,2,3,4],dtype=complex)
3) shape:-  it will return a number of rows and column
print(a.shape) //
a complete example of NumPy:-
import numpy as np

#a = np.array([1,2,3,4],ndmin=2)
a = np.array([[1, 2], [3, 4]])
#a = np.array([1,2,3,4],dtype=complex)
print(a)
print(a.shape)
4) numpy.zeros():-  this method will create zero values array element
 import numpy as np
print(np.zeros((2,2)))
5) numpy.ones():-   this method will create one values array elements
 import numpy as np
print(np.ones((2,2)))
6) numpy.reshape():-  it is used to transpose the matrix or multi-dimension array.
   for example, if the matrix is 2*3 then it will convert into 3*2
   numpy.reshape(a, newShape, order='C')
import numpy as np
e  = np.array([(1,2,3), (4,5,6)])
print(e)
var = e.reshape(3,2)

print(var)
var=np.reshape(e,(3,2),order='C')

print(var)
7) flatten:-  It is used to display array in column style, Multi dimension array to single dimension array we will use flatten().

numpy.flatten(order='C')

 import numpy as np

e = np.array([(1,2,3), (4,5,6)])

var=e.flatten() 

print(var)

8) hstack() and vstack():-

hstack():-  it is used to append array data horizontally and display results using a single dimension array.
a = np.array([1,2,3])
b = np.array([4,5,6])
c= numpy.hstack((a,b))
vstack():-  it is used to append array data vertically and display results into a multi-dimension array.
a = np.array([1,2,3])
b = np.array([4,5,6])
d = numpy.vstack((a,b))
Program to implement vstack and hstack:-

import numpy as np
a  = np.array([1,2,3])
b = np.array([4,7,8])
c = np.hstack((a,b))
d= np.hstack((a,b))
9) numpy.random.normal():-  It is used to display random numbers based on start, distance, and last index.
numpy.random.normal(loc, scale, size)
Here

• Loc: the mean. The center of distribution
• scale: standard deviation.
• Size: number of returns

## Generate random nmber from normal distribution
normal_array = np.random.normal(5, 0.5, 10)
print(normal_array)   
[5.56171852 4.84233558 4.65392767 4.946659   4.85165567 5.61211317 4.46704244 5.22675736 4.49888936 4.68731125]   

10) linspace():-  it is used to provide a sequence of data using starting point, ending point, and size.
     numpy.linspace(start, stop, size, endpoint)
It is used to subdivide the range data based on size
example of linespace():-
import numpy as np
a=np.linspace(0.0, 1.0, num=6)
print(a)
,........................................................................................................................

LogSpace

LogSpace returns even spaced numbers on a log scale. Logspace has the same parameters as np.linspace.

numpy.logspace(start, stop, num, endpoint)

 import numpy as np

a=np.logspace(3, 4, num=3)
print(a)

it will provide 10**3 and 10**3.5 , 10**4   10 is the default base of log

 .................................................................

 Indexing and slicing in Numpy:-

Indexing is used to display a particular element of the NumPy array.

slicing is the show data of a particular range

Q) WAP to display first row of numpy array?

import numpy as np
e  = np.array([(1,2,3), (4,5,6)])
print(e[0]) 

Another example of slicing :-

 Q)  Display Particular row in numpy array?

import numpy as np

e  = np.array([(1,2,3), (4,5,6)])
print(e[:2]) 

Q)  Display numpy array elements using index array?

import numpy as np
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9])
arr = x[np.array([1, 3, -3])]
print("\n Elements are : \n",arr)

Q)  Display numpy array elements using starting index, ending index and step

Syntax

arr[start:last:step]

Example of Display NumPy array using starting, ending and step's.

import numpy as np

arr = np.arange(20)
print(arr)
arr1 = arr[1:5:2]
print("\n Elements are : \n",arr1)

Advance Indexing:-

1)  Indexing by tuple index

We can fetch ndarray using paired index  So the index of the elements, in this case, are (0,0),(1,0),(2,1) and the corresponding elements are selected.

import numpy as np

a = np.array([[1 ,2 ],[3 ,4 ],[5 ,6 ]])

print(a)    

print(a[[0 ,1 ,2 ],[0 ,0 ,1]])

2) Indexing by boolean:-

using this we can provide condition to fetch elements from numpy array.

arr = np.array([23,67,89,11,22])
print(arr[arr>60])  

1) How to read data from excel file ?

We can read data from excel file using two different module, xlrd openpyxl .
xlrd is only used to read .xlx file extension for old xlx file
openpyxl is used to read .xls and .xlsx both

1) Example to read xlx file using xlrd module.

import xlrd
 
# Give the location of the file
loc = ("d://covidnew.xls")
 
# To open Workbook
wb = xlrd.open_workbook(loc)
sheet = wb.sheet_by_index(0)
 
# For row 0 and column 0
print(sheet.cell_value(0, 0))
print("Total no of rows is ",sheet.nrows)
print("Total no of columns is ",sheet.ncols)

data repository can be download by following link.

https://github.com/shivaconceptsolution/repository

Now i am explaining another module openpyxl to read data from xlsx file.

pip install openpyxl

Complete Program Explanation to Read Data From openpyxl

Solution:-

from openpyxl import load_workbook
wb = load_workbook('d://covid.xlsx')
ws = wb.active

ar =[]   #Empty List

col = ws.max_column

ro = ws.max_row

for i in range(1,(ro+1)):
   k=0
   ar1=[]#Empty List
   for j in range(1,(col+1)):

       c1 = ws.cell(row = i, column = j)
       ar1.append(c1.value)
   k+=1
   ar.append(ar1)

print(ar)
#arr = np.concatenate((ar))
#arr1 = np.hstack((ar))# Joining Numpy Array using hstack().
#arr2 = np.vstack((ar))# Joining Numpy Array using vstack().

NumPy Statistical Functions with Example:-

 it is used to provide in-built() to implement statistical operation using min,max,deviation,variance,median,mean

 import numpy as np
normal_array = np.random.normal(5, 0.5, 10)
print(normal_array)  
print(np.min(normal_array))

### Max
print(np.max(normal_array))

### Mean
print(np.mean(normal_array))

### Median
print(np.median(normal_array))

### Sd
print(np.std(normal_array))

Another example of statistical function:-

import numpy as np

arr = np.array([23,11,78,11,90,160])
print(np.min(arr))
print(np.max(arr))
print(np.median(arr))
print(np.mean(arr))
print(np.std(arr))
print(np.average(arr))

arr= np.array([[1,23,78],[98,60,75],[79,25,48]])  

print(arr)
print(np.amax(arr),np.amin(arr))

a = np.array([20,20,20])  
print(a)  
print(np.percentile(a,10,0))

Assignements of Numpy Statistical functions:-

ASSIGNMENTS:-

create repository using excel file that contain

rno, name, branch, semester, total marks gender 
................................................................

1)  Display Record of max obtained marks student, Min obtained marks

2)  Display agreegate perecentage of male, female and all

3)  Display agreegate percentage of branch wise and all

4)  Display agreegate percetage using branch and semester

5)  max marks for each branch and semester for male , female and all

 ........................................................................

 Numpy dot product in python:-

It is used to cross multiply matrix row elements,  it will perform multiplication similar to matmul() for one dimension and two dimensions and above 2 dimensions it will modify the shape but matmul() will not modify the shape of the result.

For matmul:

If either argument is N-D, N > 2, it is treated as a stack of matrices residing in the last two indexes and broadcast accordingly.

For np.dot:

For 2-D arrays it is equivalent to matrix multiplication, and for 1-D arrays to inner product of vectors (without complex conjugation). For N dimensions it is a sum product over the last axis of a and the second-to-last of b

NumPy Matmul():-  it is used to multiply matrix elements with row element to column elements.

import numpy as np

x = np.array([[1,2],[3,4]])

y = np.array([[3,4],[5,6]])

z= x.dot(y)

print(z)

#h = [[1,2],[3,4]]

#i = [[5,6],[7,8]]

z = np.matmul(x,y)

print(z)  

x = np.array([1,2,3])
y = np.array([4,5,6])
z = np.dot(x,y)
print(z)
z = np.matmul(x,y)
print(z)

x = np.array([[[1]]])
y = np.array([[[4]]])
z = np.dot(x,y)
print(type(z),z.shape)
z = np.matmul(x,y)
print(type(z),z.shape)

Determinant

Last but not least, if you need to compute the determinant, you can use np.linalg.det(). Note that numpy takes care of the dimension.
## Determinant 2*2 matrix ###

 i = [[5,6],[7,8]]

        5*8-7*6

np.linalg.det(i) 

 np.linalg.matrix_rank(A)):-  It return number of columns in matrix 

 np.trace(A))  :-  It return sum of main diagonal of matrix

print(np.linalg.det(A)) :-  It return determinant of matrix   

print(np.linalg.inv(A)) :-   It provide inverse of matrix

print( np.linalg.matrix_power(A, 3))  :- it will multiple the matrix three times 

M = np.array([[2, 1],
              [1, 2]])

result = np.linalg.matrix_power(M, 3)

1. • $M^{2}M^2$:

• $M^{3}M^3$:

$$\left[\begin{array}{cc}5& 4\\ 4& 5\end{array}\right]\times \left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]=\left[\begin{array}{cc}10& 7\\ 7& 10\end{array}\right]$$

Example of Determinant Formula:-

import numpy as np

0
 
print(np.linalg.matrix_rank(A))
print(np.trace(A))
print(np.linalg.det(A))
print(np.linalg.inv(A))
print( np.linalg.matrix_power(A, 3))

Numpy Eigen Function Example:-

This function is used to return the eigenvalues and eigenvectors of a complex Hermitian (conjugate symmetric) or a real symmetric matrix.

Tutorials for Eigen Value and Eigen Vector you can refer this site:-

https://www.mathsisfun.com/algebra/eigenvalue.html

eigh():-  Returns two objects, a 1-D array containing the eigenvalues of a matrix, and a 2-D square array or matrix (depending on the input type) of the corresponding eigenvectors (in columns). 

from numpy import linalg as scs

c, d = scs.eigh(a)

print("Eigen value is :", c)
print("Eigen vector  is :", d)

numpy.linalg.eig(a): 

This function is used to compute the eigenvalues and right eigenvectors of a square array.

import numpy as np
from numpy import linalg as scs

a = np.diag((1, 2, 3))
print("Array is :",a)

# calculating an eigenvalue
# using eig() function
c, d = scs.eig(a)
print("Eigen value is :",c)
print("Eigen vector is :",d)

Numpy DateTime Function:-

var = np.datetime64('2017-02-12')

...........................................................................

How to get the difference between two dates using NumPy?

import numpy as np

import datetime

from datetime import date

curdate = np.datetime64(datetime.datetime.now())

print(curdate)

dob = np.datetime64('2017-02-12')

ms=curdate-dob

print(np.timedelta64(ms,'D'))    

Get Year:-

import numpy as np

import datetime

from datetime import date

curdate = np.datetime64(datetime.datetime.now())

print(curdate)

dob = np.datetime64('2017-02-12')

ms=curdate-dob

dd=str(np.timedelta64(ms,'D'))

print(dd)    

y = list(dd.split(' '))

print(int(y[0])//365)

How to get the difference between two dates using Python?

today = datetime.date.today()

f_date = date(2014, 7, 2)

l_date = date(2021, 1, 1)

delta = today - l_date

print(delta.days)

How to work with Images on Numpy Array

pip install OpenCV-python

1) Using np.full() method : 

it is the simplest approach to generate an image.

 

import numpy as np

import cv2 array_created = np.full((500, 500, 3),198, dtype = np.uint8) # displaying the image cv2.imshow("image", array_created) cv2.waitKey()

Method 2: By creating an array using np.zeroes() :  

Example 1st:-

import numpy as np

import cv2

  

# creating array using np.zeroes()

array = np.zeros([500, 500, 3],

                 dtype = np.uint8)

#print(array)  

# setting RGB color values as 255,255,255

array[:, :] = [245, 45, 5] 

print(array)  

# displaying the image

cv2.imshow("image", array)

cv2.waitKey()

Example 2nd:-

import numpy as np

import numpy import cv2 img = numpy.zeros([5,5,3]) img[:,:,0] = numpy.ones([5,5])*64/255.0 img[:,:,1] = numpy.ones([5,5])*128/255.0 img[:,:,2] = numpy.ones([5,5])*192/255.0 cv2.imwrite('color_img.jpg', img) cv2.imshow("image", img) cv2.waitKey()

How to Convert images to NumPy array?

Images are an easier way to represent the working model. In Machine Learning, Python uses the image data in the format of Height, Width, Channel format. i.e. Images are converted into Numpy Array in Height, Width, Channel format. Modules Needed: NumPy: By default in higher versions of Python like 3.x onwards,

NumPy is available and if not available(in lower versions),

one can install by using pip install numpy

Pillow: This has to be explicitly installed in later versions too.

It is a preferred image manipulation tool. In Python 3, Pillow python library which is nothing but the upgradation of PIL only.

It can be installed using

pip install Pillow

One can easily check the version of installed Pillow by using the below code import PIL print('Installed Pillow Version:', PIL.__version__) Output: Installed Pillow Version: 7.2.0 Loading the images via Pillow Library Let us check for an image that is in the PNG or JPEG format.

The image can be referred via its path.

Image class is the heart of PIL. It has open() function which opens up

an image and digital file format can be retrieved as well as pixel format. from PIL import Image # sample.png is the name of the image # file and assuming that it is uploaded # in the current directory or we need # to give the path image = Image.open('Sample.png') # summarize some details about the image print(image.format) print(image.size) print(image.mode)

Converting an image into NumPy Array

Python provides many modules and API for converting an image into a NumPy

array. Let’s discuss a few of them in detail. Using NumPy module The Numpy module in itself provides various methods to do the same.

These methods are – Method 1: Using asarray() function asarray() function is used to convert PIL images into NumPy arrays.

This function converts the input to an array # Import the necessary libraries from PIL import Image from numpy import asarray # load the image and convert into # numpy array img = Image.open('Sample.png') # asarray() class is used to convert # PIL images into NumPy arrays numpydata = asarray(img) # <class 'numpy.ndarray'> print(type(numpydata)) # shape print(numpydata.shape) Output : <class 'numpy.ndarray'> (200, 400, 3)

Method 2: Using numpy.array() function By using numpy.array() function which takes an image as the argument and

converts to NumPy array from PIL import Image import numpy img= Image.open("Sample.png") np_img = numpy.array(img) print(np_img.shape) Output : (200, 400, 3) In order to get the value of each pixel of the NumPy array image,

we need to print the retrieved data that got either from asarray() function

or array() function. # Import the necessary libraries from PIL import Image from numpy import asarray # load the image and convert into # numpy array img = Image.open('Sample.png') numpydata = asarray(img) # data print(numpydata)

Another Example of cv2 to show images on the different color combinations:-

import cv2

import numpy as np

img=cv2.imread("D:\\a.jpg")

cv2.imshow("output",img)

cv2.waitKey(0)

B,G,R=cv2.split(img)

zeros=np.zeros(img.shape[:2],dtype="uint8")

cv2.imshow("Red",cv2.merge([zeros,zeros,R]))

cv2.imshow("Green",cv2.merge([zeros,G,zeros]))

cv2.imshow("Blue",cv2.merge([B,zeros,zeros]))

cv2.waitKey(0)

ASSIGNMENTS:-

CONVERT JSON FILE TO NUMPY ARRAY?

CONVERT REMOTE JSON https://shivaconceptsolution.com/webservices/showreg.php

to NUMPY ARRAY?

CONVERT CSV TO ARRAY?

Compare two images that are equal or not?

Solution of This Assignment:-

import cv2

import numpy as np

img=cv2.imread('D:\\a.jpg')

numpydata =np.array(img)

img1=cv2.imread('D:\\b.jpg') #7997

numpydata1=np.array(img1)

if np.all(numpydata==numpydata1):

    print("same image")

else:

    print("not same")

cv2.imshow("open",img)

cv2.waitKey(0)

cv2.imshow("open1",img1)

cv2.waitKey(0)

Another Example:-

import cv2

img=cv2.imread("d:\\a.jpg")

img_hsv=cv2.cvtColor(img,cv2.COLOR_BGR2HSV)

cv2.imshow("hsv",img_hsv)

cv2.imshow("hue:-",img_hsv[:,:,0])

cv2.imshow("saturation:-",img_hsv[:,:,1])

cv2.imshow("value:-",img_hsv[:,:,2])

cv2.waitKey(0)

Another Example of PIL

import PIL
import numpy as np
img=Image.open('C:\\Users\DELL\\Pictures\\cdgi\\FB_IMG_1475301504207.jpg')
numpydata =np.array(img)
img1=Image.open('C:\\Users\DELL\\Pictures\\cdgi\\FB_IMG_1475301507997.jpg')
numpydata1=np.array(img1)
if np.all(numpydata==numpydata1):
print("same image")
else:
print("not same")

If you want to do more practice on Image Processing then

 you can visit cv2 site.

https://www.analyticsvidhya.com/blog/2021/05

/image-processing-using-opencv-with-practical-examples/

How to read text from Image File Example?

pip install opencv -python

pip install pytesseract

https://osdn.net/projects/sfnet_tesseract-ocr-alt/downloads/

tesseract-ocr-setup-3.02.02.exe/

import cv2 import pytesseract # Mention the installed location of Tesseract-OCR in your system pytesseract.pytesseract.tesseract_cmd = 'C:\\Program Files (x86)\\

Tesseract-OCR\\tesseract.exe' # Read image from which text needs to be extracted img = cv2.imread("d://sample4.jpg") # Preprocessing the image starts # Convert the image to gray scale gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Performing OTSU threshold ret, thresh1 = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU | cv2.THRESH_BINARY_INV) # Specify structure shape and kernel size. # Kernel size increases or decreases the area # of the rectangle to be detected. # A smaller value like (10, 10) will detect # each word instead of a sentence. rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (18, 18)) # Appplying dilation on the threshold image dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1) # Finding contours contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Creating a copy of image im2 = img.copy() # A text file is created and flushed file = open("recognized.txt", "w+") file.write("") file.close() # Looping through the identified contours # Then rectangular part is cropped and passed on # to pytesseract for extracting text from it # Extracted text is then written into the text file for cnt in contours: x, y, w, h = cv2.boundingRect(cnt) # Drawing a rectangle on copied image rect = cv2.rectangle(im2, (x, y), (x + w, y + h), (0, 255, 0), 2) # Cropping the text block for giving input to OCR cropped = im2[y:y + h, x:x + w] # Open the file in append mode file = open("d:\\recognized.txt", "a") # Apply OCR on the cropped image text = pytesseract.image_to_string(cropped) # Appending the text into file file.write(text) file.write("\n") # Close the file file.close

Maximum Size

Maximum size images, minium sizes

Image to text conversion and text to speech conversion in data science?

1) first install two different module

1.1) pip install pyttsx3

1.2) pip install gtts

1.3) download

https://osdn.net/projects/sfnet_tesseract-ocr-alt/

downloads/tesseract-ocr-setup-3.02.02.exe/

.exe file and install it and provide their path.

Complete code of this

import pytesseract from PIL import Image import os from gtts import gTTS import pyttsx3 img = Image.open('d://sample_image.png') print(img) pytesseract.pytesseract.tesseract_cmd ='C://Program Files (x86)

//Tesseract-OCR//tesseract.exe' result = pytesseract.image_to_string(img) with open('d://abc.txt',mode ='w+') as file: file.write(result) print(result) engine = pyttsx3.init() engine.say(result) engine.runAndWait() rate = engine.getProperty("rate") print(rate) engine.setProperty("rate", 300) engine.say(result) engine.runAndWait() engine.setProperty("rate", 100) engine.say(result) engine.runAndWait() ReplyDelete

For Web Scrapping Visit this site.

https://realpython.com/beautiful-soup-web-scraper-python/

Web scrapping means to get raw data from website using web url.

Example 1st:-

from urllib.request import urlopen

page = urlopen('http://olympus.realpython.org/profiles/aphrodite')

page

html_bytes = page.read()

html = html_bytes.decode("utf-8")

html

Example2:-

title_index = html.find("<title>")

start_index = title_index + len("<title>")

end_index = html.find("</title>")

title = html[start_index:end_index]

title

Get title using regular expression:-

import re

title_pattern = r'<title>(.*?)</title>'

title_match = re.search(title_pattern, html, re.IGNORECASE)

title_match.group(1)

Example4:-

pip install beautifulsoup4

Web Scarping using HTML Parser:-

# beauty_soup.py

from bs4 import BeautifulSoup

from urllib.request import urlopen

url = "https://facebook.com/"

page = urlopen(url)

html = page.read().decode("utf-8")

soup = BeautifulSoup(html, "html.parser")

soup

print(soup.get_text())

Example 5:-

Submit Login form using Web Scraping:-

pip install mechanicalsoup

import mechanicalsoup

browser = mechanicalsoup.Browser()

url = "http://olympus.realpython.org/login"

login_page = browser.get(url)

login_html = login_page.soup

# 2

form = login_html.select("form")[0]

form.select("input")[0]["value"] = "zeus"

form.select("input")[1]["value"] = "ThunderDude"

# 3

profiles_page = browser.submit(form, login_page.url)

print(profiles_page.url)

NUMPY Predefine Array?

all()

any()

take()

put()

apply_along_axis()

apply_over_axes()

argmin()

argmax()

nanargmin()

nanargmax()

amax()

amin()

insert()

delete()

append()

around()

flip()

fliplr()

flipud()

triu()

tril()

tri()

empty()

empty_like()

zeros()

zeros_like()

ones()

ones_like()

full_like()

diag()

diagflat()

diag_indices()

asmatrix()

bmat()

eye()

roll()

identity()

arange()

place()

extract()

compress()

rot90()

tile()

reshape()

ravel()

isinf()

isrealobj()

isscalar()

isneginf()

isposinf()

iscomplex()

isnan()

iscomplexobj()

isreal()

isfinite()

isfortran()

exp()

exp2()

fix()

hypot()

absolute()

ceil()

floor()

degrees()

radians()

npv()

fv()

pv()

power()

float_power()

log()

log1()

log2()

log10()

dot()

vdot()

trunc()

divide()

floor_divide()

true_divide()

random.rand()

random.randn()

ndarray.flat()

expm1()

bincount()

rint()

equal()

not_equal()

less()

less_equal()

greater()

greater_equal()

prod()

square()

cbrt()

logical_or()

logical_and()

logical_not()

logical_xor()

array_equal()

array_equiv()

sin()

cos()

tan()

sinh()

cosh()

tanh()

arcsin()

arccos()

arctan()

arctan2()

Assignments of the Numpy Array?

MOST Important Array Interview Questions for Interview?

Level 1:-

Check if a key is present in every segment of size k in an array?

Find the minimum and maximum element in an array

Write a program to reverse the array

Write a program to sort the given array

Find the Kth largest and Kth smallest number in an array

Find the occurrence of an integer in the array

Sort the array of 0s, 1s, and 2s

Range and Coefficient of array

Move all the negative elements to one side of the array

Find the Union and Intersection of the two sorted arrays

Level 2

Write a program to cyclically rotate an array by one

Find the missing integer

Count Pairs with given sum

Find duplicates in an array

Sort an Array using the Quicksort algorithm

Find common elements in three sorted arrays

Find the first repeating element in an array of integers

Find the first non-repeating element in a given array of integers

Find the largest three elements in an array Time

Rearrange the array in alternating positive and negative items

Find if there is any subarray with sum equal to zero

Find Largest sum contiguous Subarray

Find the factorial of a large number

Find Maximum Product Subarray

Find longest consecutive subsequence

Find the minimum element in a rotated and sorted array

Find all elements that appear more than N/K times

GCD of given index ranges in an array

Minimize the maximum difference between the heights

Minimum number of jumps to reach the end

Find the two repetitive elements in a given array

Find a triplet that sums to a given value

Construct a N*M matrix from the user input

Find the row with the maximum number of 1’s

Print the matrix in a Spiral manner

Find whether an array is a subset of another array

Implement two Stacks in an array

Majority Element

Wave Array

Trapping Rainwater

Level 3

Maximum Index

Max sum path in two arrays

Find Missing And Repeating

Stock buy and sell Problem

Pair with given sum in a sorted array

Chocolate Distribution Problem

Longest Consecutive Subsequence

Print all possible combinations of r elements in a given array

How to Write data on xlrd?

import xlwt

EXCEL_FILES_FOLDER = 'd:/'

workbook = xlwt.Workbook()

worksheet = workbook.add_sheet('Student')

excel_file_path = EXCEL_FILES_FOLDER+'write_excel.xls'

def addDataToSheet(worksheet):

    # first row

    worksheet.write(1, 0, "2001")

    worksheet.write(1, 1, "James")

    worksheet.write(1, 2, "Computer")

    worksheet.write(1, 3, "A")

    # Second row

    worksheet.write(2, 0, "2002")

    worksheet.write(2, 1, "Jhones")

    worksheet.write(2, 2, "Electronics")

    worksheet.write(2, 3, "A+")

    # Third row

    worksheet.write(3, 0, "2003")

    worksheet.write(3, 1, "Micheal")

    worksheet.write(3, 2, "Civil")

    worksheet.write(3, 3, "C")

addDataToSheet(worksheet)

workbook.save(excel_file_path)

Read data from xlrd?

import xlrd

import xlwt

loc = ("d://Book1.xls")

 

# To open Workbook

wb = xlrd.open_workbook(loc)

sheet = wb.sheet_by_index(0)

sheet.write(0,2,30) 

wb.save()

# For row 0 and column 0

print(sheet.cell_value(0, 0))

print("Total no of rows is ",sheet.nrows)

print("Total no of columns is ",sheet.ncols)