How can I delete the repeated data in MYSQL with over ten million data in it?

seems to work fine I would say that there are a wide variety of benefits to using a database over a CSV for such large structured data so I would suggest that you learn enough to do so. However, based on your description you might want to check out non-server/lighter weight databases. Such as SQLite, or something similar to JavaDB/Derby... or depending on the structure of your data a non-relational (Nosql) database- obviously you will need one with some type of python support though.

will help you From the documentation of unique, we find that the unique(B, 'rows') command will
output unique rows only sort the output

B = [...
    your data here
];

C = unique(B, 'rows');

I wish this helpful for you Continuing on your table path. I assign your table to an object. The names of the desired table entries are then extracted and used to subset the data frame.

tt <- table(A$x_1)
A[!A$x_1 %in% names(tt[tt == 1]), ]

# or
A[A$x_1 %in% names(tt[tt > 1]), ]

#     x_1      z_1       z_2
# 1    A1 69.18667 0.8578626
# 2    A1 71.36819 2.8482506
# 3    A1 69.71246 1.9528315
# 4   B10 69.47145 1.7852872
# 5   B10 69.12699 0.7663739
# 6   B10 70.93589 1.1431804
# 7   B10 68.72273 0.6836297
# 9  C100 70.31252 2.4651336
# 10 C100 69.89168 1.9991948
# 11 C100 70.25079 1.0823843
# 13 G100 69.56992 2.0879085
# 14 G100 68.29589 2.5432109

Any of those help I have a database with one particular table having more than 4 million record entries. I tried downloading whole db it using MySQL workbench as well as command terminal using following command: , Try adding the below lines in the my.cnf and restart

[mysqld]
# Performance settings used for import.
delay_key_write=ALL
bulk_insert_buffer_size=256M

mysqldump -u root -p --max_allowed_packet=1073741824 --lock-tables=false mydb > myfile.sql

it helps some times LOAD DATA INFILE is your solution. You can read the documentation from the MySQL website and generate the LOAD DATA query for your need. Make sure you put the file in a place where MySQL process can read. It can only load files from certain location. Again it is part of the documentation.
https://dev.mysql.com/doc/refman/8.0/en/load-data.html