Selenium tut for scraping

I'm not that good at java, I just used it enough to know what i'm doing (when it comes to selenium)

Hidden Content

You must

[To see links please register here]

or

[To see links please register here]

to view this content.

W/o comments

Hidden Content

You must

[To see links please register here]

or

[To see links please register here]

to view this content.

I need to know, can I use Selenium on any PC after creating a java application? Or I need to install Selenium on each PC I intend to use the application on?

I can't get a great look at the code since I'm on mobile, but you should methodize the program (i.e., use methods to split it into chunks) and let it take command line arguments (the String[] args in main) to scrape the user's desired site. Also, with sites using couldflare, you need to give it a few seconds to pass the DDoS protection page.

---

(08-26-2016, 09:03 AM)BORW3 Wrote:

[To see links please register here]

I need to know, can I use Selenium on any PC after creating a java application? Or I need to install Selenium on each PC I intend to use the application on?

After compiling ("creating", in your terms) the program, it will work on any computer you use it on. That's why Java requires import statements for everything: import statements tell the compiler what to include in the final product. Additionally, since Java is completely cross-platform, you can run it on any computer that has Java installed, not just PC.

(08-26-2016, 04:54 PM)Ao- Wrote:

[To see links please register here]

I can't get a great look at the code since I'm on mobile, but you should methodize the program (i.e., use methods to split it into chunks) and let it take command line arguments (the String[] args in main) to scrape the user's desired site. Also, with sites using couldflare, you need to give it a few seconds to pass the DDoS protection page.

---

(08-26-2016, 09:03 AM)BORW3 Wrote:

[To see links please register here]

I need to know, can I use Selenium on any PC after creating a java application? Or I need to install Selenium on each PC I intend to use the application on?

After compiling ("creating", in your terms) the program, it will work on any computer you use it on. That's why Java requires import statements for everything: import statements tell the compiler what to include in the final product. Additionally, since Java is completely cross-platform, you can run it on any computer that has Java installed, not just PC.

Does Selenium support javascript input on submit buttons? Because I tried using HTMLUnit and it didn't submit my requests on clicking buttons with it, I read I found that if the website had javascript errors then it would'nt load and work properly. So Does Selenium have this same problem?

(08-27-2016, 12:05 PM)BORW3 Wrote:

[To see links please register here]

(08-26-2016, 04:54 PM)Ao- Wrote:

[To see links please register here]

I can't get a great look at the code since I'm on mobile, but you should methodize the program (i.e., use methods to split it into chunks) and let it take command line arguments (the String[] args in main) to scrape the user's desired site. Also, with sites using couldflare, you need to give it a few seconds to pass the DDoS protection page.

---

(08-26-2016, 09:03 AM)BORW3 Wrote:

[To see links please register here]

I need to know, can I use Selenium on any PC after creating a java application? Or I need to install Selenium on each PC I intend to use the application on?

After compiling ("creating", in your terms) the program, it will work on any computer you use it on. That's why Java requires import statements for everything: import statements tell the compiler what to include in the final product. Additionally, since Java is completely cross-platform, you can run it on any computer that has Java installed, not just PC.

Does Selenium support javascript input on submit buttons? Because I tried using HTMLUnit and it didn't submit my requests on clicking buttons with it, I read I found that if the website had javascript errors then it would'nt load and work properly. So Does Selenium have this same problem?

I mainly use the Python bindings for selenium, so I'm not sure if it's a Java-specific issue, but any JavaScript events/actions/listeners should work if they're put together correctly

(08-27-2016, 02:33 PM)Ao- Wrote:

[To see links please register here]

(08-27-2016, 12:05 PM)BORW3 Wrote:

[To see links please register here]

(08-26-2016, 04:54 PM)Ao- Wrote:

[To see links please register here]

I can't get a great look at the code since I'm on mobile, but you should methodize the program (i.e., use methods to split it into chunks) and let it take command line arguments (the String[] args in main) to scrape the user's desired site. Also, with sites using couldflare, you need to give it a few seconds to pass the DDoS protection page.

---

After compiling ("creating", in your terms) the program, it will work on any computer you use it on. That's why Java requires import statements for everything: import statements tell the compiler what to include in the final product. Additionally, since Java is completely cross-platform, you can run it on any computer that has Java installed, not just PC.

Does Selenium support javascript input on submit buttons? Because I tried using HTMLUnit and it didn't submit my requests on clicking buttons with it, I read I found that if the website had javascript errors then it would'nt load and work properly. So Does Selenium have this same problem?

I mainly use the Python bindings for selenium, so I'm not sure if it's a Java-specific issue, but any JavaScript events/actions/listeners should work if they're put together correctly

Okay, I have tried it, works better with javascript than HTMLUnit ever did. I also tried creating a youtube bot to rack up views with Selenium. I used HtmlUNitDriver wih Selenium and a get() to a video page, i didn't add the web views, but opening the link with firefox normally worked fine. So.... Wondering if you can assist me here.

(08-27-2016, 12:05 PM)BORW3 Wrote:

[To see links please register here]

(08-26-2016, 04:54 PM)Ao- Wrote:

[To see links please register here]

I can't get a great look at the code since I'm on mobile, but you should methodize the program (i.e., use methods to split it into chunks) and let it take command line arguments (the String[] args in main) to scrape the user's desired site. Also, with sites using couldflare, you need to give it a few seconds to pass the DDoS protection page.

---

(08-26-2016, 09:03 AM)BORW3 Wrote:

[To see links please register here]

I need to know, can I use Selenium on any PC after creating a java application? Or I need to install Selenium on each PC I intend to use the application on?

After compiling ("creating", in your terms) the program, it will work on any computer you use it on. That's why Java requires import statements for everything: import statements tell the compiler what to include in the final product. Additionally, since Java is completely cross-platform, you can run it on any computer that has Java installed, not just PC.

Does Selenium support javascript input on submit buttons? Because I tried using HTMLUnit and it didn't submit my requests on clicking buttons with it, I read I found that if the website had javascript errors then it would'nt load and work properly. So Does Selenium have this same problem?

Are you trying to Click a Submit button via JS?

(08-29-2016, 01:54 PM)BORW3 Wrote:

[To see links please register here]

(08-27-2016, 02:33 PM)Ao- Wrote:

[To see links please register here]

(08-27-2016, 12:05 PM)BORW3 Wrote:

[To see links please register here]

Does Selenium support javascript input on submit buttons? Because I tried using HTMLUnit and it didn't submit my requests on clicking buttons with it, I read I found that if the website had javascript errors then it would'nt load and work properly. So Does Selenium have this same problem?

I mainly use the Python bindings for selenium, so I'm not sure if it's a Java-specific issue, but any JavaScript events/actions/listeners should work if they're put together correctly

Okay, I have tried it, works better with javascript than HTMLUnit ever did. I also tried creating a youtube bot to rack up views with Selenium. I used HtmlUNitDriver wih Selenium and a get() to a video page, i didn't add the web views, but opening the link with firefox normally worked fine. So.... Wondering if you can assist me here.

Numberphile did a good video on this topic. YouTube tries their best to veto "counterfeit" views, so it's harder than just visiting the page.

You are making a pretty big mistake here. NEVER assume an element exists in Selenium, if anything goes wrong it will exception out and you will be screwed.

(08-29-2016, 04:22 PM)Ao- Wrote:

[To see links please register here]

(08-29-2016, 01:54 PM)BORW3 Wrote:

[To see links please register here]

(08-27-2016, 02:33 PM)Ao- Wrote:

[To see links please register here]

I mainly use the Python bindings for selenium, so I'm not sure if it's a Java-specific issue, but any JavaScript events/actions/listeners should work if they're put together correctly

Okay, I have tried it, works better with javascript than HTMLUnit ever did. I also tried creating a youtube bot to rack up views with Selenium. I used HtmlUNitDriver wih Selenium and a get() to a video page, i didn't add the web views, but opening the link with firefox normally worked fine. So.... Wondering if you can assist me here.

Numberphile did a good video on this topic. YouTube tries their best to veto "counterfeit" views, so it's harder than just visiting the page.

LOL. Man. 301 is end of bot counters.