logo
down
shadow

Jsoup- getting certain attributes from website


Jsoup- getting certain attributes from website

Content Index :

Jsoup- getting certain attributes from website
Tag : java , By : Adam Hill
Date : November 28 2020, 01:01 AM


Comments
No Comments Right Now !

Boards Message :
You Must Login Or Sign Up to Add Your Comments .

Share : facebook icon twitter icon

Fetching the website with Jsoup - page view source and Jsoup shows different content


Tag : java , By : ChaseVoid
Date : March 29 2020, 07:55 AM
hope this fix your issue I use Jsoup to scrap the website: , Short answer Jsoup can't execute the Javascript.
Long answer
http://www.yelp.com/search?find_desc=restaurant&find_loc=willowbrook%2C+IL&ns=1#l=p:IL:Willowbrook::&sortby=rating&rpp=40

Getting element with no attributes using Jsoup


Tag : java , By : wraith
Date : March 29 2020, 07:55 AM
it should still fix some issue I think you can use the JSOUP CSS selector p:not([^]), which would select any p that does not match having an attribute starting with anything.
String html = "<div id=\"intro\">"
        + "<h1 class=\"some class\">"
        + "<p id=\"some_id\">"
        +   "Some text 1"
        + "</p>"
        + "<p name=\"some_name\">"
        +   "Some text A"
        + "</p>"
        + "<p data>"
        +   "Some text B"
        + "</p>"
        +"<p>"
        +   "Some text 2"
        +"</p>"
        +"</div> ";

Document doc = Jsoup.parse(html);
Elements els = doc.select("p:not([^])");
for (Element el:els){
    System.out.println(el.text());
}
Some text 2

Get Some Attributes with JSoup


Tag : java , By : zdyne
Date : November 17 2020, 09:01 AM
I wish this helpful for you I was having some practices with programming, and I got stuck (also because of my lacking knowledge of web programming) in this part: I was to get some information from this page: http://db.fowtcg.us/index.php?p=card&code=VS01-003+R , but only the card properties, and I'm struggling a little with JSoup, I was able to fetch the data with: , Here, use this instead:
Elements property = doc.select("div.col-xs-12.col-sm-7.box.card-props");

method filling array with Jsoup not waiting for jsoup to complete website request


Tag : java , By : user179271
Date : March 29 2020, 07:55 AM
wish helps you Jsoup.connect().get() is a synchronous call, so when it returns it will have connected and retrieved the response. The issue in your code - which you have correctly identifed as being somehow related to 'waiting for something to finish' - is because you invoke Jsoup.connect().get() inside a separate thread and then you do not wait for that thread to complete before attempting to use what Jsoup returns.
At this line:
.start()
for (int i = rawHours.size() - 1; i >= 0; i--) {
    hrrrLabels[23 - i] = rawHours.get(i);
}
...
Thread t = new Thread(new Runnable() {
    ...
});
t.start();

// wait for the 'Jsoup thread' to complete before continuing
t.join();

for (int i = rawHours.size() - 1; i >= 0; i--) {
    hrrrLabels[23 - i] = rawHours.get(i);
}
...
hrrrLabels = new String[24];
final LinkedList<String> rawHours = new LinkedList<>();
final StringBuilder builder = new StringBuilder();

try {
    Document doc =
            Jsoup.connect("http://mag.ncep.noaa.gov/model-guidance-model-parameter.php?group=Model%20Guidance&model=HRRR&area=CONUS&ps=model").get();

    Elements links = doc.select("tr");
    int superi = 0;
    for (int i = 22; i < 26; i++) {
        Element link = links.get(i);
        Elements lin = link.select("td");
        Element time;

        for (int j = 0; j < lin.size(); j++) {
            time = lin.get(j);
            rawHours.add(time.text());
            builder.append(time.text() + "\n");

        }
        superi++;
    }
} catch (IOException e) {
    builder.append("Error :      ").append(e.getMessage()).append("\n");
}

for (int i = rawHours.size() - 1; i >= 0; i--) {
    hrrrLabels[23 - i] = rawHours.get(i);
}
String[] SplitTime;
String[] hrrrTimes = new String[hrrrLabels.length];
System.out.println("rewtimes, length=" + hrrrLabels.length);
for (int i = 0; i < hrrrLabels.length; i++) {
    System.out.println("rewtimes, i=" + i + " :" + hrrrTimes[i]);
    SplitTime = hrrrLabels[i].split(" ");
    hrrrTimes[i] = SplitTime[1].substring(0, 2);
}

Problem scraping website using Java Jsoup, website not "scrolling"


Tag : java , By : user183275
Date : March 29 2020, 07:55 AM
Hope that helps I would suggest opening the browser's developer tab for trying to find out which url/endpoint the website uses for fetching new items for the infinite scroll, as JSoup does not execute Javascript itself. Then you can call the endpoint with JSoup and parse the results.
In case it does not work, It would be probably better to move to HtmlUnit or Selenium as both of them are full-featured web browser APIs which you can control with Java.
Related Posts Related QUESTIONS :
  • Instance variable heap or stack ? ( with specific example)
  • Assert progress of ProgressBar in Espresso test
  • How to detect if gson.fromjson() has excess elements
  • I cant generate the proper code to select the a specific filter on a BI dashboard I am working on
  • How to Inject Dependencies into a Servlet Filter with Spring Boot Filter Registration Bean?
  • Thrift types as a Generic
  • Effective algorithm to random 4 unique integers less than a big max such as 100_000
  • Combining or and negation in Java regex?
  • Unable to instantiate default tuplizer Exception
  • Multi-tenant migration to work with quarkus
  • Ignite persisting a Set: Cannot find metadata for object with compact footer
  • Maven cannot resolve Jacob dependency using eclipse
  • testcontainers oracle database container starts before database user is created
  • Launching two spring boot apps in integration test
  • Is there a way to add a HashMap's value that is a integer array into a ArrayList?
  • Is there any way that I can get a parameter in paintComponent?
  • Empty stack with one recursive method and one iterative method
  • What's the behavior of onBackpressureBuffer in RxJava2
  • Java regex can only use 1 quantifier in a lookback (need 2)
  • How to fix error in native query : it is showing syntax error near or at
  • How to retrieve nested object from a document and display it in FirestoreRecyclerOptions?
  • Why not use ListIterator for full LinkedList Operation?
  • Android Webview EvaluateJavascript sometimes does not return a response
  • Matcher java doesn't work but regex seems to be good
  • Finding dimensions of a .gif file
  • Java Number format how to change +/- sign to custom text
  • Entity partially saved when using JOINED inheritance strategy and setting spring.jpa.properties.hibernate.jdbc.batch_siz
  • Stored Procedure in Java Spring Boot Project returns null as Output
  • How to solve org.hibernate.MappingException which is causing due to inheriting a class
  • Clean Archtecture. Understanding of scheme
  • Processing 3 triangle not showing in Javafx 8 Window tab
  • How to specify a sequence-based generated value in Hibernate 5 via legacy mapping
  • Spring-boot application not getting auto-deployed on startup
  • How to only pass strings that the user select
  • Is there a way to SELECT using "GREATEST(field1, field2)" where field1 and field2 are aggregate sums in the sa
  • How to handle JSON objects wrapped into one JSON object with retrofit2?
  • Configure Hazelcast CPSubsystem Retries Timeout
  • how to use onBindViewHolder with multiple items in android RecyclerView
  • No ParameterResolver registered for parameter in BeforeAll method
  • Finding the path in a graph with the least casualties according to the lanchester square law
  • MongoWriteException when inserting into Mongodb with composite custom _id
  • Fetch Oracle procedure metadata with Java when multiple procedure signatures
  • Value modification of key-pair in HashMap and impact for a HashCode
  • Migration from solrj to spring-data-solr
  • How to check if you're still connected to the database with jpa
  • Use Date type in the graphql scheme
  • Split and add the string based on length
  • Is "main" method of spring boot application required when deploy as war
  • Getting the average within specific numbers in an array
  • how to use izpack to make my jar application to installer?
  • What is meant by src in Java Eclipse?
  • Create a mirrored linked list in Java
  • Examples of good JPA Java Desktop Application
  • Translate Java to Python -- signing strings with PEM certificate files
  • Algorithm Analysis tool for java
  • Java serial comm API - what does inputstream.read() return if a timeout occurs?
  • How do I make a background thread in Java that allows the main application to exit completely? This works in Linux, but
  • How to add an image dynamically at runtime in java
  • Java App on Mac asking for allow network connections everytime
  • Best actively maintained Java XMPP Library?
  • shadow
    Privacy Policy - Terms - Contact Us © scrbit.com