Site scraping import sites.

JLennox

JLennox

CAG Veteran
I have created a program to scrape the price lists from several importer's sites then to generate a list of prices which have changed. I intent to post this information in a single thread with one post per morning, as well as having an ftp script automaticly upload an alternitive version to my web host.

I would perfer it if people do not post information which is irrelivent to every one else, such as, "I MISSED IT!" I do not want this to turn into a Target thread where each page has 1 informitive post, and the remainder being ramble.

For the time being, I would perfer non-community (as in, ideas directed at me) discussion to take place through my email- jlennox[at]gmail[dt]com

I do fully intend to use refeeral links, and I'm sure no one is blaming me. So far, I have only made it compatable with YesAsia's GBA and DS sections, and here is the output going from yesterday into today:

NEW Playasia_ds
Yugiou Dual Monster NIGHTMARE TROUBADOUR Japan Version
Now: $47.99

Rockman Exe 5 DS Twin Readers Japan Version
Now: $44.99

Fullmetal Alchemist Dual Sympathy Japan Version
Now: $44.99

Rockman Exe DS Pouch Japan Version
Now: $27.99

Playasia_ds
Nintendo DS Active Pouch Black) (Japan Version
Now: $35.99 Before: $19.99

Nintendo DS Active Pouch Blue) (Japan Version
Now: $35.99 Before: $19.99

NEW Playasia_gba
BLEACH Advance Akani Somaru Shikonkai Japan Version
Now: $45.99

Lilo and Stitch Japan Version
Now: $43.99

Kim Possible Japan Version
Now: $43.99

Playasia_gba
Sonic Advance 2 Japan Version
Now: $54.99 Before: $37.99

I am attempting to track if there is an actual interest in this?
 
[quote name='JLennox']
I do fully intend to use refeeral links, and I'm sure no one is blaming me. [/QUOTE]

You mean your own referral links that you get credit for? I think someone here might blame you.
 
[quote name='fwacce']You mean your own referral links that you get credit for? I think someone here might blame you.[/QUOTE]

Oh? I was unaware getting store credit for your efforts was problem-matic. I am aware that I do not pay the bills around here, nor did I have any part in the creation of this community who I intend to gain this credit from, but I do not picture an opperator having trouble with this.

If I actualy get a care'able sum of credit (10$+), which I'm not counting on, I'll donate a percentage to CheappyD, but as said, I do not picture taking in even that amount.

My goal is essentialy a cheap-ass-o-matic for import sites. They'll often mark items down with out putting it on their front page or so forth, nor do I wish to check their front pages on a daily basis.

I'm not going to post any additional sales until I get the program into a more ready state or until any problem is resolved if the above mentioned issue is indeed troublesome. I also intend to release the source code (C#) once I feel it's ready. Though I would ask for you to not cut in on my buisness, but modify it for your own.
 
Don't bother. As soon as the retailers finds out you are running a script on their sites (and they will, its only a matter of time) they will shut you down. Just ask Rockhero...his script was shutdown by EB in less than 2 weeks (i think).

And yes, you are not allowed to use your own referral links on this site.
moved to deal discussions
 
I think it's a good idea, but this is Cheapy's site, and so referral links automatically become his.
 
I'm not worried about them restricting my ip address.

I'd be more then willing to do some thing to the extent of a 60% chance of the site's referral showing up and a 40% chance of my own, but otherwise, despite how much I'd like to be charitible, I'd lose interest in the day to day effort required, quickly.
 
Could you update your title to say 'scraping' instead of 'scrapping' I finally clicked it wondering what sort of imports the sites are scrapping or ??? :lol:

Sounds like you'll probably be scrapping the whole idea of creating this scraper anyway, now that cheapy has weighed in.
 
[quote name='wubb']Could you update your title to say 'scraping' instead of 'scrapping' I finally clicked it wondering what sort of imports the sites are scrapping or ??? :lol:[/QUOTE]

Sorry, my English is poor, though it is my only language. Fixed now, though.

[quote name='wubb']Sounds like you'll probably be scrapping the whole idea of creating this scraper anyway, now that cheapy has weighed in.[/QUOTE]

If we don't come to a compramise and I can't find any other community that would be willing to do so, I'll most likely do it for a bit on here, then see if I can ditch the effort onto some one who has the time/desire to be more charitble then I.

If I do find another community which would allow for this, I wount have much trouble re-posting here with Cheappy's referral- seeing it would most likely not hurt buy-throughs in a care'able amount, and it would be a minimal amount of effort since the rest will already be finished.

I wouldn't want to "go out on my own" due to the lack of a pre-existing community and the lack of desire in attempting to build one for such a trivial thing.
 
You must log in or register to reply here.
bread's done
Back
Top