Find Jobs
Hire Freelancers

Scape broken as cookie and csrf changed. Needs fixing

$30-250 USD

Complété
Publié il y a environ 2 mois

$30-250 USD

Payé lors de la livraison
I have a python scrape that collects data from a website but the cookie and csrf have changed and I am not sure how to obtain the ones from dev tools. Here is the top part of the broken scrape code. Can you fix it? Full code attached. # Scraping [login to view URL] import os import datetime from time import sleep import requests import pyodbc from [login to view URL] import InsecureRequestWarning requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning) def terminate(error_message, code, data, d_type): print(f"[{d_type}]{error_message}, Check Logs. Exiting...") dt = str([login to view URL]()).replace(".", "_").replace(":", "_").replace(" ", "_").replace("-", "_") print(f"\nPlease Check File: scraper_logs/{dt}.txt\n") with open(f"scraper_logs/{dt}.txt", "w", encoding="utf-8") as logs: [login to view URL](f"[{[login to view URL]()}]\n\n[Status Code]: {code}\n\n[Response]: {data}") exit() def parse_commit(db_conn, db_cursor, dtype, data): dtype_data = data[dtype] dtype_id = dtype_data["id"] dtype_name = dtype_data["name"] dtype_url = urls[f"{dtype}_url"] + dtype_id params = (dtype_id, dtype_name, dtype_url) if dtype == "series": [login to view URL](f"SELECT id FROM series WHERE series_id='{dtype_id}'") db_data = [login to view URL]() if db_data: return db_data[0], dtype_data["season"] else: [login to view URL](f'''INSERT INTO series(series_id, series_name, series_url) VALUES (?, ?, ?)''', params) [login to view URL]() [login to view URL]("SELECT @@IDENTITY") return [login to view URL](), dtype_data["season"] [login to view URL](f"SELECT id FROM {dtype} WHERE {dtype}_id='{dtype_id}'") db_data = [login to view URL]() if db_data: return db_data[0] [login to view URL](f'''INSERT INTO {dtype}({dtype}_id, {dtype}_name, {dtype}_url) VALUES (?, ?, ?)''', params) [login to view URL]() [login to view URL]("SELECT @@IDENTITY") return [login to view URL]() if not [login to view URL]("scraper_logs/"): [login to view URL]("scraper_logs") urls = { "collectible_url": "[login to view URL]", "series_url": "[login to view URL]", "licensor_url": "[login to view URL]", "brand_url": "[login to view URL]", } editions = { "TI": "Trial Issue", "AP": "Artist Proof", "FA": "First Appearance", "FE": "First Edition", "CE": "Con Exclusive", "RE": "Retail Exclusive", "EE": "Event Exclusive", "PR": "Promotion", "VE": "VE-VE Exclusive" } rarities = { "COMMON": 1, "UNCOMMON": 2, "RARE": 3, "ULTRA_RARE": 4, "SECRET_RARE": 5, "ARTIST_PROOF": 6 } cookies = { 'veve': 's%[login to view URL]', } headers = { 'client-operation': 'MarketByCollectibleTypeQuery', 'csrf-token': 'fQgHwfAM-mVpHbYIQxNhB1ADH5s9HWNpdEtg', } headers1 = { 'client-operation': 'MarketFromCollectibleTypeQuery', 'csrf-token': 'fQgHwfAM-mVpHbYIQxNhB1ADH5s9HWNpdEtg', } # Connect to SQL Server server = '.\MSSQLSERVER01' etc etc etc
N° de projet : 37932596

Concernant le projet

24 propositions
Projet à distance
Actif à il y a 1 mois

Cherchez-vous à gagner de l'argent ?

Avantages de faire une offre sur Freelancer

Fixez votre budget et vos délais
Soyez payé pour votre travail
Surlignez votre proposition
Il est gratuit de s'inscrire et de faire des offres sur des travaux
Décerné à :
Avatar de l'utilisateur
I have the skills and expertise necessary to fix your broken Python scrape. I have a strong command over Python and web scraping techniques which are at the core of your project.I can quickly and effectively adapt your code to obtain the new cookies and csrf from dev tools, ensuring a smooth retrieval of data from the target website. My proficiency with data extraction from websites coupled with my knack for delivering structured, and organized results in the format of your preference makes me an ideal candidate for this project. Moreover, I have cross-domain experience in utilizing headers such as 'client-operation' and 'csrf-token' which can be a valuable asset for troubleshooting and streamlining your scrape. Lastly, my keen eye for detail, strong communication skills, and knack for problem-solving will ensure that your technical issues are addressed promptly and effectively. I understand the importance of delivering desired results in a timely manner, leaving no room for errors or ambiguities. Let's fix those bugs together and get your scrape back on track!
$100 USD en 1 jour
5,0 (4 commentaires)
3,0
3,0
24 freelances proposent en moyenne $145 USD pour ce travail
Avatar de l'utilisateur
I am a Python web scraping expert with experience in handling website changes such as cookies and CSRF tokens. I can fix your broken scrape code and ensure smooth data collection from the specified website. With attention to detail and accuracy, I will provide an efficient solution to address the new requirements. Let's discuss further details and costs.
$250 USD en 2 jours
5,0 (180 commentaires)
7,9
7,9
Avatar de l'utilisateur
Hello Employer, I have seen your attached code. I am an expert in Python web scraping projects and I am interested and available to work on the project. Kindly share the project details and requirements. You can send me a message to know about the solution to your project. Thank you, Looking forward to hearing from you.
$100 USD en 7 jours
5,0 (39 commentaires)
6,2
6,2
Avatar de l'utilisateur
Hello Greetings, After reviewing your project description, I feel confident and excited to work on this project for you. But I have some crucial things and queries to clear out. Can you please leave a message on chat so we can discuss this and I can share my recent work similar to your requirements? Thanks for your time! I look forward to hearing from you soon. Best Regards
$250 USD en 4 jours
5,0 (6 commentaires)
5,6
5,6
Avatar de l'utilisateur
As an experienced freelance python developer and web scraper, I have all the requisite skills to get your scraping code back on track in no time. I understand how frustrating it can be when the scraper code fails, but fear not, I specialize in quickly and effectively locating and resolving issues like yours. With over X years of experience handling similar problems with various websites, I'm adept at adapting to any website shift, including dev tools. What sets me apart is my dedication and attachment towards clean coding standards which make future modifications easier. In addition to fixing your project's immediate issue, I can also suggest long-term optimizations for it. I have detailed working knowledge with libraries like Requests and BeautifulSoup which will come in handy while remediating your project. Trust me with this project and you won't be disappointed; my goal is to ensure your valuable data scraping is up and running reliably again. Let's schedule a chat so we can dive right into fixing this code!
$140 USD en 3 jours
4,8 (21 commentaires)
5,2
5,2
Avatar de l'utilisateur
(((((((( Available to start working immediately )))))))) I understand your situation. When website elements like cookies and CSRF tokens change, it can disrupt existing scraping scripts. However, I can help you adapt your Python scraper to obtain the new cookies and CSRF tokens from the website's developer tools. By utilizing browser developer tools, we can inspect network requests to identify the updated cookies and CSRF tokens required for successful scraping. I'll implement a solution in your Python scraper to dynamically retrieve and utilize these elements, ensuring seamless data collection from the website. With my experience in web scraping and Python programming, I'm confident that I can quickly resolve this issue and get your scraper back up and running efficiently. Let's discuss further details and get started on resolving this challenge.
$113 USD en 1 jour
5,0 (19 commentaires)
4,8
4,8
Avatar de l'utilisateur
Hello, I can start immediately. I have 13+ years of experience and I ensure you I can complete the project within budget and timeline. I'm not just a developer; I'm your dedicated partner. I'm committed to working with you. Count on me for any upgrades or specific requirements you may have along the way. I have few questions about the requirement. Can we discuss this over chat? Looking forward to hearing from you soon. Best regards, Sneha
$213 USD en 7 jours
5,0 (5 commentaires)
4,5
4,5
Avatar de l'utilisateur
Hi there,Good evening I am Talha. I can work with your project skills Web Scraping and Python I am excited to present my proposal, which centers around a personalized approach designed to elevate your project. We will start with an in-depth consultation to gain a deep understanding of your project's unique requirements, goals, and constraints. Our commitment to customization means that we will tailor our services to align perfectly with your project, and we will explain how this approach will meet your expectations. Please note that the initial bid is an estimate, and the final quote will be provided after a thorough discussion of the project requirements or upon reviewing any detailed documentation you can share. Could you please share any available detailed documentation? I'm also open to further discussions to explore specific aspects of the project. Thanks for considering my proposal. I'm eager to collaborate and contribute to your project's success. Let me know if you need any more information. Regards. Talha Ramzan
$30 USD en 11 jours
5,0 (3 commentaires)
3,8
3,8
Avatar de l'utilisateur
⭐ Hi, My availability is immediate. I read your project post on Python Developer. We are experienced full-stack Python developers with skill sets in - Python, Django, Flask, FastAPI, Jupyter Notebook, Selenium, Data Visualization, ETL - Web App Development, Data Science, Web/API Scrapping - API Development, Authentication, Authorization - SQLAlchemy, PostegresDB, MySQL, SQLite, SQLServer, Datasets - Web hosting, Docker, Azure, AWS, GPC, Digital Ocean, GoDaddy, Web Hosting - Python Libraries: NumPy, pandas, scikit-learn, tensorflow, etc. - ML Toos: ChatGPT, Llama, Google Bard, OpenAI, Artificial Intelligence, - AWS SageMaker, AWS Bedrock, AWS Machine Learning Services, AWS AI Services - Azure Cognitive Services, Azure Bot Service, Azure QnA Maker, Azure Vision, Azure Document Intelligence, Azure OpenAI - Tableau, PowerBI - AI: Generative AI, Langchain, LLM, RAG - Artificial Intelligence, Machine Learning, Deep Learning, Chatbot Please send a message So we can quickly discuss your project and proceed further. I am looking forward to hearing from you. Thanks
$240 USD en 3 jours
4,3 (14 commentaires)
4,1
4,1
Avatar de l'utilisateur
Hi there, I believe I can help you with this project as i have the experience with Python and Web Scraping. here is my portfolio: https://www.freelancer.com/u/ayesha0124
$250 USD en 9 jours
5,0 (1 commentaire)
3,0
3,0
Avatar de l'utilisateur
Hello, How are you? I appreciate the opportunity to submit this proposal and am confident in my ability to deliver a high-quality services that meets your objectives. I look forward to discussing this proposal further and addressing any questions or concerns you may have. Please feel free to contact me to discuss any additional details. I am excited about the possibility of working together to bring this project to fruition. Thank you for considering my proposal. I am eager to contribute to the success of your business. Sincerely
$150 USD en 2 jours
5,0 (4 commentaires)
3,1
3,1
Avatar de l'utilisateur
I can fix the broken scrape code for you. With expertise in Python, Data Collection, and Web Scraping, rest assured the project will be handled with precision. Our 24*7 support ensures quick assistance. Let's discuss more to expedite the process.
$30 USD en 7 jours
5,0 (2 commentaires)
0,9
0,9
Avatar de l'utilisateur
Hi James N., Are you looking for a talented developer to bring your digital projects to life? Look no further! As an expert in software development, I possess a diverse skill set that goes beyond coding languages. With a passion for technology and a drive for excellence, I am committed to delivering high-quality work on time and within budget. My expertise includes proficiency in Web Scraping and Python, enabling me to create stunning and responsive web applications. Additionally, I am well-versed in mobile development frameworks such as Flutter, React Native, Android, and iOS (including Swift), ensuring that your applications are compatible across multiple platforms. But it's not just about technical expertise – I also excel in communication, collaboration, and project management. I am dedicated to understanding your unique needs and delivering solutions that exceed your expectations. Whether you need a website, a mobile app, or a custom software solution, I am your partner in innovation. Please reach me out and make your project live.
$155 USD en 4 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
Hi James N., With a proven track record of delivering exceptional results, I bring a unique blend of skills, expertise, and passion to every project I undertake. As a talented developer, I possess a deep understanding of various programming languages, frameworks, and technologies such as Web Scraping and Python. Whether it's web development, mobile app development, or software engineering, I have the versatility and proficiency to tackle diverse challenges with precision and efficiency. What sets me apart is my commitment to excellence and innovation. I thrive on pushing the boundaries of what's possible, leveraging cutting-edge tools and techniques to deliver solutions that exceed expectations. From crafting elegant code to designing intuitive user experiences, I approach every aspect of development with meticulous attention to detail and a relentless pursuit of excellence. Moreover, my strong communication skills and collaborative mindset enable me to seamlessly integrate with your team, understand your project requirements, and deliver results that align with your vision and objectives. Whether you're a startup looking to build a minimum viable product or an enterprise seeking to modernize your technology stack, I have the expertise and dedication to help you achieve your goals. Please reach me out on chat.
$155 USD en 6 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
Hi James N., As an expert in software development, I am well-versed in a wide range of technologies and frameworks, including Web Scraping and Python. With a keen eye for detail and a passion for creating seamless user experiences, I excel at crafting responsive and visually appealing websites and mobile applications. Whether you need a stunning website to showcase your brand or a feature-rich mobile app to engage your users, I have the skills and expertise to deliver top-notch results. In addition to my technical skills, I am also a strong communicator and collaborator. I believe in working closely with my clients to understand their unique requirements and goals, ensuring that the final product meets their expectations. From initial concept to final deployment, I am committed to providing exceptional service and support every step of the way. So if you're looking for a talented developer who can turn your vision into reality, I'm here to help. Let's work together to bring your digital projects to life and achieve your business objectives.
$155 USD en 6 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
Hi,i read and analyse the code carefully. There are issue in your headers and some configuration issue is also persist there. Let's connect so that we can fix it. I'm looking forward for your response
$145 USD en 7 jours
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
@@@@@ It's a piece of cake @@@@@ Hello. Thank you for your job posting. I just read your job posting and it sounds like you need Python and Web scraping expert. I can do it within a short time with high quality perfectly as I have full experience with Python and Web scraping. If you give me a chance, I will do my best to show good result in an day. Looking forward to hearing from you soon. Best Regards, Slavko
$100 USD en 1 jour
0,0 (0 commentaires)
0,0
0,0
Avatar de l'utilisateur
As an expert at Techfusion, I have extensive experience in Python and web scraping, making me the perfect fit for your project. I understand the frustration of encountering change in source code requirements, but worry not! As a seasoned developer, I've adapted to countless similar situations and overcome them each time. Moreover, my skills extend beyond mere scraping; I specialize in cloud architecture and data analytics as well. This can prove crucial in retrieving the new cookie and CSRF information that you need from dev tools. My proficiency also includes the use of requests library and managing with csrf tokens, all of which are crucial aspects to fix your broken code. When it comes to problem-solving, I am thorough and efficient. I will meticulously evaluate your existing code, identify the changes required due to new cookies and csrf fields and swiftly deliver a solution that covers all bases. Choose me for this project and rest assured that your scrape will once again be running smoothly, collecting data without any hitches! Note: Please note that initial bid is estimated and the final quote will be provided after discussion of the project requirements
$30 USD en 7 jours
0,0 (0 commentaires)
1,0
1,0
Avatar de l'utilisateur
===================Scraping Expert=================== Hello, I have reviewed your job description and I can do your project I have rich experience in web scraping using Selenium, Scrapy and BeautifulSoup. Please contact me via chat to discuss your project Thank you
$150 USD en 2 jours
0,0 (0 commentaires)
0,0
0,0

À propos du client

Drapeau de UNITED KINGDOM
Farnborough, United Kingdom
5,0
35
Méthode de paiement vérifiée
Membre depuis juin 29, 2015

Vérification du client

Merci ! Nous vous avons envoyé un lien par e-mail afin de réclamer votre crédit gratuit.
Une erreur a eu lieu lors de l'envoi de votre e-mail. Veuillez réessayer.
Utilisateurs enregistrés Total des travaux publiés
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Chargement de l'aperçu
Permission donnée pour la géolocalisation.
Votre session de connexion a expiré et vous avez été déconnecté. Veuillez vous connecter à nouveau.