DataBloom

NVIDIA Modulus is now available on NVIDIA LaunchPad. Sign-up for a free, hands-on lab that will teach you how to develop physics-informed machine-learning…

NVIDIA Modulus is now available on NVIDIA LaunchPad. Sign-up for a free, hands-on lab that will teach you how to develop physics-informed machine-learning solutions.

HDBSCAN is a state-of-the-art, density-based clustering algorithm that has become popular in domains as varied as topic modeling, genomics, and geospatial…

HDBSCAN is a state-of-the-art, density-based clustering algorithm that has become popular in domains as varied as topic modeling, genomics, and geospatial analytics.

RAPIDS cuML has provided accelerated HDBSCAN since the 21.10 release in October 2021, as detailed in GPU-Accelerated Hierarchical DBSCAN with RAPIDS cuML – Let’s Get Back To The Future. However, support for soft clustering (also known as fuzzy clustering) was not included. With soft clustering, a vector of values (rather than a single cluster label) is created for each point representing the probability that the point is a member of each cluster. 

Performing HDBSCAN soft clustering on CPUs has been slow. It can take hours or even days on medium-sized datasets due to the heavy computational burden. Now, in the 22.10 RAPIDS release, cuML provides accelerated soft clustering for HDBSCAN, enabling the use of this technique on large datasets.

This post highlights the importance of using soft clustering to better capture nuance in downstream analysis and the performance gains possible with RAPIDS. In a document clustering example, soft clustering that takes hours on a CPU can be completed in seconds with cuML on a GPU.

Soft clustering code

In the CPU-based scikit-learn-contrib/hdbscan library, soft clustering is available through the all_points_membership_vectors top-level module function. 

cuML matches this API:

import cuml

blobs, labels = cuml.datasets.make_blobs(n_samples=2000, n_features=10, random_state=12)

clusterer = cuml.cluster.hdbscan.HDBSCAN(prediction_data=True)
clusterer.fit(blobs)
cuml.cluster.hdbscan.all_points_membership_vectors(clusterer)

Why soft clustering

In many clustering algorithms, each record in a dataset is either assigned to a single cluster or considered noise and not assigned to any clusters. However, in the real world, many records do not perfectly fit into a single cluster. HDBSCAN acknowledges this reality by providing a mechanism for soft clustering by representing the degree to which each point belongs to each cluster. This is similar to other algorithms such as mixture models and fuzzy c-means.

Imagine a news article about a sports-themed musical. If you wanted to assign the article to a cluster, would it belong to a sports cluster or a musical cluster? Perhaps a smaller cluster specifically for sports-themed musicals? Or should it have some level of membership in both clusters?

By forcing the choice of a single label, this nuance is lost, which can be significant when the results of clustering are used for downstream analysis or actions. If only a sports label is assigned, would a recommendation system surface the article to readers also interested in musicals? What about the inverse (a reader interested in one but not the other)? This article should be potentially in the queue for readers interested in either or both of these topics.

Soft clustering solves this problem, enabling actions based on thresholds for each category and creating applications that provide a better experience.

Example of document clustering

You can use document clustering to measure the potential real-world performance benefits and impact of cuML’s accelerated soft clustering. 

Many modern document clustering workflows are composed of the following steps:

• Convert each document into a numeric representation (often using neural network embeddings)
• Reduce the dimensionality of the numeric-transformed documents
• Perform clustering on the dimension-reduced dataset
• Take actions based on the results

If you would like to run the workflow on your system, a Jupyter Notebook is available by visiting hdbscan-blog.ipynb on GitHub.

Preparing the dataset

This example uses the A Million News Headlines dataset from Kaggle, which contains over 1 million news article headlines from the Australian Broadcasting Corporation. 

After downloading the dataset, convert each headline into an embedding vector. To do this, use the all-MiniLM-L6-v2 neural network from the Sentence Transformers library. Note that a few other libraries imported will be used later.

import numpy as np
import pandas as pd
import cuml
from sentence_transformers import SentenceTransformer

N_HEADLINES = 10000

model = SentenceTransformer('all-MiniLM-L6-v2')

df = pd.read_csv("/path/to/million-headlines.zip")
embeddings = model.encode(df.headline_text[:N_HEADLINES])

Reducing dimensionality

With the embeddings ready, reduce the dimensionality down to five features using cuML’s UMAP. This example is only using 10,000 records, as it is intended as a guide.

For reproducibility, set a random_state. The results in this workflow may differ slightly when run on your machine, depending on the UMAP results.

umap = cuml.manifold.UMAP(n_components=5, n_neighbors=15, min_dist=0.0, random_state=12)
reduced_data = umap.fit_transform(embeddings)

Soft clustering

Next, fit the HDBSCAN model on the dataset and enable using all_points_membership_vectors for soft clustering by setting prediction_data=True. Also set min_cluster_size=50, which means that groupings of fewer than 50 headlines will be considered as part of another cluster (or noise), rather than as a separate cluster.

clusterer = cuml.cluster.hdbscan.HDBSCAN(min_cluster_size=50, metric='euclidean', prediction_data=True)
clusterer.fit(reduced_data)
soft_clusters = cuml.cluster.hdbscan.all_points_membership_vectors(clusterer)
soft_clusters[0]
array([0.01466704, 0.01035215, 0.0220814 , 0.01829496, 0.0127591 ,
       0.02333117, 0.01993877, 0.02453639, 0.03369896, 0.02514531,
       0.05555269, 0.04149485, 0.05131698, 0.02297594, 0.03559102,
       0.02765776, 0.04131499, 0.06404213, 0.01866449, 0.01557038,
       0.01696391], dtype=float32)

Inspecting a couple of clusters

Before going further, look at the assigned labels (-1 represents noise). It is evident that there are quite a few clusters in this data:

pd.Series(clusterer.labels_).value_counts()
-1     3943
 3     1988
 5     1022
 20     682
 13     367
 7      210
 0      199
 2      192
 8      190
 16     177
 12     161
 17     143
 15     107
 19      86
 9       83
 11      70
 4       68
 18      66
 6       65
 10      61
 1       61
 14      59
dtype: int64

Select two clusters at random, clusters 7 and 10, for example, and examine a few points from each.

df[:N_HEADLINES].loc[clusterer.labels_ == 7].headline_text.head()
572    man accused of selling bali bomb chemicals goe...
671      timor sea treaty will be ratfied shortly martin
678     us asks indonesia to improve human rights record
797           shop owner on trial accused over bali bomb
874    gatecrashers blamed for violence at bali thank...
Name: headline_text, dtype: object

df[:N_HEADLINES].loc[clusterer.labels_ == 10].headline_text.head()
40     direct anger at govt not soldiers crean urges
94                   mayor warns landfill protesters
334                    more anti war rallies planned
362     pm criticism of protesters disgraceful crean
363      pm defends criticism of anti war protesters
Name: headline_text, dtype: object

Then, use the soft clustering membership scores to find some points, which might belong to both of the clusters. From the soft cluster scores, identify the top two clusters for each point. Exclude outliers by filtering for soft memberships to clusters 7 and 10, which are both proportionately larger than their memberships to other clusters:

df2 = pd.DataFrame(soft_clusters.argsort()[:,::-1][:,:2])
df2["sum"] = soft_clusters.sum(axis=1)
df2["cluster_7_ratio"] = soft_clusters[:,7] / df2["sum"]
df2["cluster_10_ratio"] = soft_clusters[:,10] / df2["sum"]
df2[(df2[0] == 7) & (df2[1] == 10) & (df2["cluster_7_ratio"] > 0.1) & (df2["cluster_10_ratio"] > 0.1)]

3824  7  10  0.630313         0.170000         0.160453
6405  7  10  0.695286         0.260162         0.119036

Inspect the headlines for these points, and notice they are both about Indonesia, which is also in the cluster 7 headline 678 (above). Also notice that both of these headlines are about antiwar and peace, which are topics included in several of the cluster 10 headlines.

df[:N_HEADLINES].iloc[3824]
publish_date                                              20030309
headline_text    indonesians stage mass prayer against war in iraq
Name: 3824, dtype: object

df[:N_HEADLINES].iloc[6405]
publish_date                           20030322
headline_text    anti war fury sweeps indonesia
Name: 6405, dtype: object

Why soft clustering matters

How confident should you be that these results belong to their assigned cluster, rather than another cluster? As previously observed, some clusters contain headlines that have a meaningful probability of being in a different cluster.

The degree of confidence can be partially quantified by calculating the difference between the membership probabilities of each point’s top two clusters. This example excludes noise points, to drive home how this does not just happen with “noisy” points but also those assigned cluster labels:

soft_non_noise = soft_clusters[clusterer.labels_ != -1]
probs_top2_non_noise = np.take_along_axis(soft_non_noise, soft_non_noise.argsort(), axis=1)[:, -2:]
diffs = np.diff(probs_top2_non_noise).ravel()

Plotting a histogram and the empirical cumulative distribution function of these differences shows that many points were close to being assigned a different cluster label. In fact, about 30% of points had top-two cluster membership probabilities within 0.2 of one another (Figure 1). 

Using these soft clustering probabilities enables you to incorporate this uncertainty to build much more robust machine learning pipelines and applications.

Performance benchmark results

Next, run the preceding core workload on different numbers of news article headlines, varying from 25,000 to 400,000 rows. Note that HDBSCAN parameters can be tweaked for larger datasets. 

For this example, CPU benchmarks were run on an x86 Intel Xeon Gold 6128 CPU at 3.40 GHz. GPU benchmarks were recorded on an NVIDIA Quadro RTX 8000 with 48 GB of memory.

On the CPU (indicated by the hdbscan backend), soft clustering performance scales loosely linearly as the number of documents is doubled. 50,000 documents took 50 seconds; 100,000 documents took 500 seconds; 200,000 documents took 5,500 seconds (1.5 hours); and 400,000 documents took over 60,000 seconds (17 hours). See Table 1 for more details.

Using the GPU-accelerated soft clustering in cuML, soft clusters for 400,000 documents can be calculated in less than 2 seconds, rather than 17 hours.

Backend	Number of Rows	Soft Clustering Time (s)
cuml	25,000	0.008182
hdbscan	25,000	5.795254
cuml	50,000	0.014839
hdbscan	50,000	53.847145
cuml	100,000	0.077507
hdbscan	100,000	485.847746
cuml	200,000	0.322825
hdbscan	200,000	5503.697239
cuml	400,000	1.343359
hdbscan	400,000	62428.348942

If you would like to run this benchmark on your system, use the benchmark-membership-vectors.py GitHub gist. Note that performance will vary depending on the CPU and GPU used.

Key takeaways

We are excited to report these performance results. Soft clustering can meaningfully improve workflows powered by machine learning. Until now, using a clustering technique like HDBSCAN has been computationally challenging for even a few hundred thousand records.

With the addition of HDBSCAN soft clustering, RAPIDS and cuML continue to break through barriers and make state-of-the-art computational techniques more accessible at scale. 

To get started with cuML, visit the RAPIDS Getting Started page, where conda packages, pip packages, and Docker containers are available. cuML is also available in the NVIDIA optimized PyTorch and Tensorflow Docker containers on NVIDIA NGC, making this end-to-end workflow even easier.

Acknowledgments

This post describes cuML functionality contributed by Tarang Jain during his internship at NVIDIA under the mentorship of Corey Nolet.

X-ray-powered research is aiming to target sneaky hazardous materials making their way through airport security. The study, recently published in Scientific…

X-ray-powered research is aiming to target sneaky hazardous materials making their way through airport security. The study, recently published in Scientific Reports, proposes a new design for a fast and powerful X-ray diffraction (XRD) technology able to identify potential threats. The work could be a notable step toward more accurate luggage scanning in airports. 

“The main goal of my project was to speed up this new X-ray imaging modality so it can be economically viable for airport security. Ultimately such a scanner could help find even the most creatively hidden explosives, drugs, and contraband, without excessive costs for the operator or delays for passengers,” said study author Airidas Korolkovas. He conducted part of the study while working as an X-ray physicist and imaging scientist at iTomography Corporation.

An awardee of the NVIDIA Academic Hardware Grant Program, Korolkovas was granted an NVIDIA TITAN V during his postdoctoral fellowship at Uppsala University in Sweden. Applicants must demonstrate how access to world-class computing resources could boost their research. 

In this case, Korolkovas designed and implemented a GPU-accelerated tomographic reconstruction algorithm for XRD to supplement existing computed tomography (CT) X-ray scans in airport security.  

Currently, airports rely on X-ray transmission alone to reveal luggage contents in 3D. X-ray beams can penetrate through, absorb, or scatter depending on the composition and spatial arrangement of atoms within each material. By measuring changes in the beam at various angles, sophisticated algorithms and computer vision technology can reconstruct 3D images of the bag contents. 

This gives airport security a peek into luggage without having to touch it. However, transmission-based CT scans have limitations. 

“Standard CT is sensitive to the average density and composition of materials. It is not sensitive to the internal arrangement of atoms, which makes all the difference when identifying a benign piece of plastic from a plastic explosive, or between sugar and cocaine,” Korolkovas said. 

According to the study, XRD could be a powerful new addition when scanning luggage, because it is sensitive to the internal arrangement of atoms.

This makes XRD especially well suited for identifying crystals as the repetitive arrangement of molecules in crystalline materials results in concentrated X-ray scattering along precise angles unique to each material. Access to this data could help airport security determine if objects in a bag contain threats like cocaine, crystal methamphetamine, or even explosives that are naturally crystalline, semi-crystalline, or crystalline powders.

Unfortunately, XRD scans of whole passenger luggage are quite slow, making them unusable in commercial aviation, which demands real-time results. 

To shorten the time, Korolkovas employed a novel scanner design aimed at high-intensity rather than high-resolution X-ray beams, which is traditionally a no-go as it degrades the XRD signal quality beyond recognition. 

Through a multistep approach, involving CT image segmentation and complex algebraic reconstructions, he was able to recover the XRD resolution, despite the limitations of beam intensity.

Korolkovas used the NVIDIA TITAN V to calculate the probabilities of all possible diffraction pathways.

“In this study, I was able to maintain acceptable resolution by combining data from transmission, diffraction, all the viewing angles, and the full spectrum of X-ray energies,” he said.

This can easily run into a quintillion mathematical operations for every slice of the bag.

According to Korolkovas, coding the reconstruction algorithm on a GPU was very helpful in keeping the computation time manageable. Changing from a CPU to a GPU, he was able to speed up the runtime from 10 hours to less than 1 hour. By further improving the algorithm and using multiple GPUs or cloud computing, he envisions eventually running the scan in real time. 

“X-rays can penetrate and scatter within the bag in every possible direction. CUDA texture mapping has turned out to be an efficient way to access the photon survival probabilities along any such pathway. The calculations of various pathways are partially independent of each other, and benefit from parallel computing afforded by CUDA,” he said.

Testing the approach on a simulated bag containing both benign and threat materials, he found that the XRD reconstruction adds material-specific information, improving what CT alone captures and threat detection. 

“An XRD imaging add-on to existing CT scanners is feasible and is well positioned to provide unique, material-specific information, at a low cost of installing an extra detector or two and developing suitable reconstruction software,” Korolkovas writes in the study. 

The next steps in the research include building an experimental prototype and testing the algorithm on real-world data. 

“The study has received encouraging feedback from Rapiscan Systems, a major manufacturer of X-ray scanners. Now that air travel is returning to pre-pandemic levels, there is renewed interest in advanced X-ray imaging and I hope to contribute to this endeavor,” Korolkovas said. 

He also plans on using machine learning to train neural networks that fingerprint the reconstructed diffraction patterns against a broad range of materials found in suitcases. This will improve the robustness of flagging threat materials, even with limited data that can be acquired in real time.

Read the study, Fast X-ray diffraction (XRD) tomography for enhanced identification of materials.
Access research code on the XRD_Tomography GitHub page.
Learn more about NVIDIA Higher Education and Research Developer Resources.

Funding for this research includes a grant from the U.S. Department of Homeland Security, Science, and Technology Directorate and a Titan V donated by NVIDIA.

The NVIDIA Optical Flow SDK 4.0 is now available, enabling you to fully harness the new NVIDIA Optical Flow Accelerator on the NVIDIA Ada architecture with…

The NVIDIA Optical Flow SDK 4.0 is now available, enabling you to fully harness the new NVIDIA Optical Flow Accelerator on the NVIDIA Ada architecture with NvOFFRUC.

Optical flow on the NVIDIA Ada Lovelace architecture

Starting from the NVIDIA Turing architecture, NVIDIA GPUs have dedicated hardware for optical flow computation between a pair of frames. NVIDIA has continued to invest in improving the optical flow hardware engine in the NVIDIA Ampere architecture and NVIDIA Ada Lovelace architecture generations, thanks to the continued feedback from application developers and researchers.

Significant performance improvements

The Optical Flow algorithm requires certain pre– and post-processing steps to improve the quality of the flow vectors.

In the NVIDIA Turing and NVIDIA Ampere architecture generation GPUs, most of these algorithms use a compute engine to perform the required tasks. As a result, when the compute engine workload is high, the performance of the NVIDIA Optical Flow Accelerator (NVOFA) could be affected.

On NVIDIA Ada-generation GPUs, most of these algorithms are moved to dedicated hardware within the NVOFA, reducing the dependency on the compute engine significantly.

In addition, NVIDIA Ada-generation GPUs bring several other optimizations related to reducing the overhead of interaction between driver and hardware. This increases the overall performance and context switches between various hardware engines on the GPU.

With these changes, the speed of the NVIDIA Ada Lovelace architecture NVOFA is improved ~2x compared to the NVIDIA Ampere architecture NVOFA.

Quality improvements

Based on the feedback from earlier generations of NVOFA, there are several quality improvements incorporated in the hardware. Using the same preset, you can see a 10-15% improvement in quality (tested on the KITTI2015 data set) compared to NVIDIA Ampere architecture GPUs.

For more information, see 1.4 NVOFA Quality and Performance.

Optical Flow SDK 4.0

The NVIDIA Optical Flow SDK enables you to access NVOFA functionality. The NVIDIA Optical Flow SDK is a set of Optical Flow C APIs, reusable C++ wrapper classes, and a set of sample applications. These APIs and C++ wrapper classes facilitate the programming of the NVOFA for the efficient computation of the optical flow between a pair of images.

Optical Flow SDK 4.0 comes with the following enhancements and features:

•  External hint support
•  NVIDIA Optical Flow-assisted Frame-Rate Up-Conversion (NvOFFRUC)

External hint support

When hints are generated with low evolution images or are available from other sources such as a game engine, NVOFA can refine the hints further to improve the quality of the flow vectors.

Though external hint support is already available through C-API, support was missing in earlier versions of SDK C++ wrapper classes.

Optical Flow SDK 4.0 adds necessary support in the C++ classes and the use of external hints is demonstrated in the sample application AppOFCuda. The hint format is the same as the output flow vector format: an array of NV_OF_FLOW_VECTOR structures. Each array element represents a motion vector for the corresponding block in raster scan order.

AppOFCuda accepts hints in Middlebury flo format but converts them into the required format (an array of NV_OF_FLOW_VECTOR structures) before passing it to the NVOF API. NVOFA prioritizes external hints when they are provided; you are expected to provide reasonable quality hints.

Frame-rate up-conversion (FRUC) is a technique that generates higher frame-rate video from lower frame-rate video by inserting interpolated frames into it. Such high frame-rate video shows smooth continuity of motion across frames, improving the perceived visual quality of the video.

The NvOFFRUC library exposes APIs that take two consecutive frames and generate an interpolated frame in between. The interpolation is instant and does not have to be exactly in the middle of the two frames: it can be specified arbitrarily. For more information, see the NVOFA FRUC Programming Guide.

These APIs can be used for up-conversion of any video content. Internally, the library uses the NVOFA hardware engine and CUDA compute cores. As a result, frame interpolation using the NvOFFRUC library is much faster compared to software-only methods.

For more information, see AV1 Encoding and FRUC: Video Performance Boosts and Higher Fidelity on the NVIDIA Ada Lovelace Architecture.

Optical Flow SDK 4.0 is available now.

The NVIDIA Base Command Platform enables an intuitive, fully featured development experience for AI applications. It was built to serve the needs of…

The NVIDIA Base Command Platform enables an intuitive, fully featured development experience for AI applications. It was built to serve the needs of the internal NVIDIA research and product development teams. Now, it has become an essential method for accessing on-demand compute resources to train neural network models and execute other accelerated computing experiments. 

Base Command Platform simplifies AI experimentation workflows by providing a cohesive service that integrates users, jobs, and data. It provides easy access to a private registry to host custom containers as well as the breadth of software from the NGC Catalog. It offers all these features without sacrificing reliable NVIDIA performance, flexibility, and scalability. You can use Base Command Platform for experiments requiring a single GPU or a data center’s worth of them.

Base Command Platform interface and features

Base Command Platform supports a CLI, API, and web interface, all built into the NGC portal. The integrated web interface makes software discovery in the NGC Catalog and subsequent use in Base Command Platform smooth. You don’t have to transition between tools not designed to be used together.

In addition to providing access to the public NGC Catalog, you also gain access to a private registry dedicated to the Base Command Platform environment. The private registry is useful for keeping containers, models, and software private and secure, as dictated by developer requirements.

Base Command Platform provides a rich set of user management controls. When you are invited to use a Base Command Platform environment (called an organization), the administrator can restrict your ability to upload and interact with content on the platform through a set of role-based access controls. These controls can apply to the root organization and also to the concept of a team. 

A team can differ minimally or significantly from the root organization, depending on how an admin configures that team. For example, a team may only be provided access to a subset of private registry containers or resources. When onboarded to a team, you could be disallowed from uploading your own containers to the private registry. 

These capabilities can be mixed and matched to provide the right level of functionality for a given user or group by the org administrator. Administrators can also set hardware usage quotas in the organization for specific users, both GPU and storage capacity.

The Base Command Platform web interface places the key user interaction points front and (left of) center: 

• Jobs: A list of containers running on NVIDIA Base Command Platform compute resources.
• Datasets: Read-only data inputs that can be mounted into jobs.
• Workspaces: Read/write persistent storage that can also be mounted into jobs.

Simple yet powerful hardware abstraction

In Base Command Platform, the managed hardware resources are presented to the user through two concepts: accelerated computing environments (ACEs) and instances within an ACE. 

An ACE is a composition of a set of hardware resources: compute, network, and storage. An instance selects the CPU, RAM, and GPU resource quantities that a job requires from a system within an ACE. 

ACEs can support a variety of instance types depending on their underlying hardware composition. Administrators can restrict the use of these resources through a quota for GPU hours, as well as completely restricting instance type availability for specific users in the org. 

Base Command Platform resources are connected through industry-leading technology provided by the underlying infrastructure. NVIDIA NVLink, NVIDIA InfiniBand, and high-performance Ethernet connectivity are integrated as part of a Base Command Platform environment’s design to maximize the value of the managed hardware resources. 

The scheduler in Base Command Platform is designed to take advantage of topology awareness to provide optimal resource use for jobs as they are submitted.

Datasets and workspaces

Data management is core to Base Command Platform’s capabilities. Datasets, models, and source code must be made available to compute resources during experimentation. 

The dataset and workspace concepts are how Base Command Platform solves this problem. A dataset is a read-only storage construct after creation, with all the same sharing capabilities as private registry contents. They can be private to a specific user, shared with any number of teams in an org, or shared with the entire org. 

Workspaces are more flexible. They are readable and writable but can be marked read-only when used in a job if desired. Workspace-sharing capabilities are identical to what datasets support.

Datasets are created through either the web interface or the CLI at upload or conversion time. We cover conversion when jobs are discussed later in this post. A workspace is created first, then populated with data as part of a job, or through direct upload (similar to datasets).

So, why would you use one over the other? 

Datasets are a great fit for immutable data that must be widely shared as-is. Frequently, that is a dataset that no longer requires modification but could also include a license file or API key intended for shared use. They can be shared widely because there is no chance that they will be modified in place. 

Workspaces are a great fit as a landing place for data that is a work-in-progress: datasets, source code maintained outside a container, or even a collection of models under development. Workspaces can be shared widely but given that they are writable by default, wide sharing may require additional coordination and awareness between users.

The aggregate dataset and workspace capacity available for a given user in Base Command Platform is controlled by the user’s storage quota, set by the org administrator. You can see your storage quota along with your current storage usage on the Base Command Platform dashboard. 

There is an additional storage type, result, that factors into this capacity, which we discuss later in the context of jobs. When your quota is exceeded, you can request additional capacity if enabled by your environment administrator.

Bringing it all together in a job

Base Command Platform operationalizes data, compute resources, the NGC Catalog, and private registry contents with jobs. 

Job creation starts with resource selection. You are presented with available ACEs for use by your org and team. You may have access to more than one ACE, but a job must execute within a single ACE. 

After an ACE is selected, you can select from the available instance types in that ACE. For multi-node jobs, the only instances available are those that leverage the maximum CPU, RAM, and GPU resources available for a given system type within the selected ACE.

Choose an ACE, an instance type, and multi-node launch options, if necessary. Next, you can specify datasets and workspaces that are a part of the chosen ACE to be mounted into the target job, along with the desired mount point for each of them. Here, a workspace can be marked as read-only. 

The job’s result storage path must be specified as well. A result is a job-specific read/write storage pool intended to hold artifacts that you’d like to preserve from the job upon completion, along with stderr and stdout. As we mentioned previously, the capacity consumed by results counts against your quota.

Then, you must select a Container object and a valid container Tag. You can choose containers from the NGC Catalog as well as the private registry containers that you have permission to access. If you select a multi-node job, only containers marked in the NGC Catalog or private registry as supporting multi-node functionality are presented as options.

You now specify one or more commands, or even a service (such as JupyterLab, Visual Studio Code Server, or NVIDIA Triton Inference Server) to run inside the selected container when the job is active. If an external port is needed to expose a web interface or some other endpoint, one or more ports must be added with the Add a Container Port option.

Now that the job is specified, there are several more options available to configure: 

• Job priority level
• New job name
• How a job is capable of behaving if preempted
• Maximum runtime of the job
• Time slice interval for telemetry collection
• Custom labels

Interacting with running and finished jobs

After a job has been launched, you are redirected to a page specific to that job, where the launch details are on the Overview tab. In fact, an equivalent CLI version of the launch form is available under the Command section. 

Jobs are presented in a way that makes them easy to reproduce: either by copying a CLI representation or by cloning the job through the web interface. If ports were added to the job when launched, a URL endpoint is also available (Video 2).

Several additional tabs are present for a scheduled, running, or completed job:

• Telemetry: Key performance metrics for a job over time.
• Status History: A list of states that the job and associated replicas have been in.
• Results: A way to view the files that have been saved to a job’s results directory.
• Log: Searchable, live access to a job’s stdout output.

After a job completes, the job-specific page is still accessible and can be used as a reference for future jobs, or further debugging if something didn’t work out as intended. The results directory and log files from the job can be easily retrieved. Depending on how the job was written, the desired job artifacts could be in a workspace instead of the results directory. 

Base Command Platform provides CLI support for downloading data from a workspace. The resulting artifacts from a job can then be uploaded into Base Command Platform, either made public or kept in the private registry for the org. It provides the starting point for further experimentation in Base Command Platform or a critical component for a model deployed elsewhere.

For more information, see Managing Jobs.

MLOps integration using the NGC API

You can further augment and extend Base Command Platform capabilities with external software integration through the documented NGC API. 

The NGC API can be used for workflow integration or dashboards outside of Base Command Platform, such as third-party MLOps platforms and tools. MLOps software and service providers that have integrated their unique offerings with Base Command Platform include Weights & Biases and Rescale.

As Base Command Platform features evolve and expand, the NGC API enables new and existing software ecosystems to integrate its strengths into other purpose-built solutions.

Conclusion

Base Command Platform is one of the key NVIDIA tools for making AI infrastructure accessible to developers. To get a hands-on sense of how Base Command Platform works, NVIDIA offers a series of labs through NVIDIA LaunchPad. Some labs cover specific use cases around natural language processing and medical imaging, and others are tailored toward gaining experience with Base Command Platform capabilities. 

Learn how to use NVIDIA Base Command Platform to accelerate your containerized AI training workloads. 

Real-time natural language understanding will transform how we interact with intelligent machines and applications.

Real-time natural language understanding will transform how we interact with intelligent machines and applications.

The role of artificial intelligence (AI) in boosting performance and energy efficiency in cellular network operations is rapidly becoming clear. This is…

The role of artificial intelligence (AI) in boosting performance and energy efficiency in cellular network operations is rapidly becoming clear. This is especially the case for radio access networks (RANs), which account for over 60% of industry costs. 

This post explains how AI is transforming the 5G RAN, improving energy and cost efficiency while supporting better use of RAN computing infrastructure.

5G is in full motion 

It is now over 4 years since the first 5G networks were launched. A GSMA study projects that there will be over 1.4 billion 5G connections by 2025. The deployment of standalone 5G networks globally is also beginning to increase. To learn more, see Operators Keep Pushing Forward on 5G Standalone Networks.

In the consumer market, 5G is the default upgrade for an ubiquitous cellular communications service. For the enterprise market, 5G has the optimal combination of high performance, mobility, flexibility, and security to provide the connectivity fabric for enterprise use cases (Figure 1). 

NVIDIA is driving innovation in cellular networks with a fully programmable NVIDIA Aerial SDK for building and deploying GPU-accelerated 5G virtual radio access networks (vRANs). This is providing the building blocks for a standard public 5G network for a telco or a private 5G network implementation with AI-on-5G. 

AI is shaping the current state—and evolution—of 5G 

The role of AI in cellular network operations is growing at a fast pace. AI delivers value through the terabytes of data collected every day—from network elements to customer interactions—and through the resulting insights. These insights are related to managing increased network demand, combating cyberthreats, optimizing services, and improving the customer experience.  

AI is currently applied across different domains in cellular networks such as RAN, core network, operations support systems (OSS), business support systems (BSS), and cloud infrastructure. These AI-enabled functionalities emerged in 4G, are becoming entrenched in 5G, and will become native in 6G. 

While AI will permeate the entire value chain of the industry, its impact on the RAN will be the most profound, particularly given the disproportionate share of industry capex and opex the RAN accounts for. Accordingly, both O-RAN and 3GPP have identified and are working on AI initiatives that improve the performance, flexibility, scalability, and efficiency of the RAN. To learn more, see Embracing AI in 5G-Advanced Towards 6G: A Joint 3GPP and O-RAN Perspective.

AI is transforming the RAN in four key ways: energy savings, mobility management and optimization, load balancing, and Cloud RAN. Read on for more details about each.

Energy savings

The rapid growth of 5G deployment has coincided with a rapid increase in energy costs globally, leading to concerns about high operational costs and carbon emissions. There is the additional concern that some 5G deployments may face hard limits on how much power can be supplied, even if the owners are willing to pay. These concerns create a powerful incentive to increase operational efficiency to achieve higher power efficiencies from current and future network deployments. See Take the Green Train: NVIDIA BlueField DPUs Drive Data Center Efficiency for more details. 

The industry has been using reactive and inflexible rule-based techniques to conserve energy, such as switching on/off cells based on different thresholds of cell load. However, AI offers a proactive and adaptive approach, enabling telcos to predict energy efficiency and load in future states. AI also provides better integration between the RAN and virtualized core network functions (with User Plane Function, for example) through offloading networking, security, and RAN tasks to NVIDIA GPUs and DPUs. 

Mobility management and optimization

Mobile communications systems have the distinct ability to support handovers of devices from one access point to another. This provides service continuity, supports mobility, and optimizes performance. Expectedly, disruptions, delays, and frequency of handovers add up to inefficiencies in network performance. 

Using AI to optimize paging and predict the next cell for handovers offers a significant opportunity to improve performance for advanced features. Such features include sophisticated dual connectivity options, conditional handover, and dual active protocol stack (DAPS) handover. 

AI prediction will rely on insights about the possible movement of the device. To achieve this, the Network Data Analytics Function (NWDAF) from 3GPP SA2 provides data from the core network, applications, and the OSS to improve handover performance, predict device location and performance, and steer traffic to achieve quality network performance.

NVIDIA continues to innovate around the NVIDIA Aerial SDK to support these new expectations for data collection and the use of AI for network management. 

Load balancing 

Handovers enable mobile networks to steer traffic to balance the load across different cell sites and to improve the use of spectrum, RAN, transport, and core infrastructure. This load-balancing decision is achieved by optimizing handover parameters and decisions using current or historical load information. 

However, this task is becoming more challenging due to the use of multiple frequency bands and interworking with different RANs. Current rules will increasingly struggle to cope with fast time-varying scenarios with high mobility, and dynamic traffic patterns with a large number of connections. 

AI models perform better and can predict load to optimize critical tasks using the collection of RAN data. This will improve network performance and user experience. This is a key driver in the development of proprietary AI tools and the current push for some industry standardization to unlock this opportunity at scale. 

Cloud RAN: Colocating RAN and AI in the cloud

While Cloud RAN is not a core application of AI in itself, it is the logical extension of the preceding load-balancing discussion. In this case, a softwarized and cloud-enabled RAN can be colocated on the same cloud infrastructure with AI workloads. Boosting RAN use beyond the typical average 25% of many sites, this will support telcos to extract efficiency gains from an asset that gulps at least 60% of industry capex. 

The suitability to share the same computational resources with other AI workloads, the availability of such orthogonal AI workloads, and the ability to use AI to switch dynamically between the different workloads are key to unlocking this opportunity. 

This Cloud RAN vision will begin in the 5G era and then mature in the 6G era, as the RAN as a workload in the cloud becomes the ultimate destination. By pooling baseband computing resources into a cloud-native environment, the Cloud RAN solution delivers significant improvements in asset use for both Cloud Service Providers (CSPs) and telcos. 

CSPs can run the RAN as a workload alongside their AI workloads within their existing data center architecture while telcos can increase RAN operational efficiency by more than 2x for an estimated >25% impact on EBITDA. 

NVIDIA is shaping this evolution with the Cloud RAN solution based on NVIDIA Spectrum switch, NVIDIA H100 CNX converged accelerator, and NVIDIA Aerial SDK. To learn more, see Unlocking New Opportunities with AI Cloud Infrastructure for 5G vRAN.

The role of AI in the RAN is only part of the overall role of AI in telecom network operations. In addition to the RAN, NVIDIA is working with partners on AI-powered operations to use data insights from telco data to create new revenues and improve operational efficiency. Visit the NVIDIA Telecommunications page to learn more.

Learn the fundamental tools and techniques for accelerating C/C++ applications to run on massively parallel GPUs with CUDA in this instructor-led workshop.

Learn the fundamental tools and techniques for accelerating C/C++ applications to run on massively parallel GPUs with CUDA in this instructor-led workshop.

Data is one of the most valuable assets that a business can possess. It sits at the core of data science and data analysis: without data, they’re both…

Data is one of the most valuable assets that a business can possess. It sits at the core of data science and data analysis: without data, they’re both obsolete. Businesses that actively collect data may have a competitive advantage over those that do not. With sufficient data, organizations can better determine the cause of problems and make informed decisions.

There are scenarios where an organization may lack sufficient data to draw necessary insights. For example, a start-up almost always begins with no data. Instead of moping about their deficiencies, a better solution is to employ data acquisition techniques to help build a custom database.

This post covers a popular data acquisition technique called web scraping. You can follow along using the code in the kurtispykes/web-scraping-real-estate-data GitHub repository.

What is data acquisition?

Data acquisition (also referred to as DAQ) may be as simple as a technician logging the temperature of your oven. You can define DAQ as the process of sampling signals that measure real-world physical phenomena and converting the resulting samples into digital numerical values that a computer can interpret.

In an ideal world, we would have all data handed to us, ready for use, whenever we wanted. However, the world is far from ideal. Data acquisition techniques exist because some problems require specific data that you may not have access to at a particular time. Before any data analysis can be conducted, the data team must have sufficient data available. One technique to acquire data is web scraping.

What is web scraping?

Web scraping is a popular data acquisition technique that has become a hot topic of discussion among those with rising demands for big data. Essentially, it’s the process of extracting information from the Internet and formatting it to be easily usable in data analytics and data science pipelines.

In the past, web scraping was a manual process. The process was tedious and time-consuming, and humans are prone to error. The most common solution is to automate. Automation of web scraping enables you to speed up the process while saving money and reducing the likelihood of human error.

However, web scraping has its challenges.

Challenges of web scraping

Building your own web scraper has challenges outside of knowing how to program and understanding HTML. It is beneficial to know in advance the various obstacles that you may encounter while data scraping. Here are a few of the most common challenges that you’ll face when scraping data from the web.

robots.txt

Permissions for scraping data are usually held in a robots.txt file. This file is used to inform crawlers about the URLs that can be accessed on a website. It prevents the site from being overloaded with requests.

The first thing to check before you begin a web scraping project is whether the target website permits web scraping. Websites can decide whether to allow web scrapers on their website for web scraping purposes.

Some websites do not permit automated web scraping, typically to prevent competitors from gaining a competitive advantage and draining the server resources from the target site. It does affect the website’s performance.

You can check the robots.txt file of a website by appending /robots.txt to the domain name. For example, check Twitter’s robots.txt as follows: www.twitter.com/robots.txt.

Structural changes

UI and UX developers periodically add, remove, and undergo regular structural changes to a website to keep it up to date with the latest advancements. Web scrapers depend on the code elements of the web page at the time that the scraper is built. Thus, frequent changes to a website may result in data being lost. It’s always a good idea to keep tabs on the changes to a web page.

Also, consider that different web page designers may have different criteria for designing their pages. This means that if you plan on scraping multiple websites, you might have to build multiple scrapers, one for each website.

IP blockers, or getting banned

It is possible to be banned from a website. If your web scraper is sending an unnaturally high number of requests to a website, it’s possible for your IP address to get banned. Alternatively, the website may restrict its access to break down the scraping process.

There’s a thin line between what is considered ethical and unethical web scraping: crossing the line quickly leads to IP blocking.

CAPTCHAs

A CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) does exactly what it says in its name: distinguishes humans from bots. The problems posed by CAPTCHAs are typically logical and straightforward for a human to solve but challenging for a bot to accomplish the same feat, which prevents websites from being spammed.

There are ethical workarounds for scraping websites with CAPTCHAs. However, that discussion is beyond the scope of this post.

Honeypot traps

In a similar fashion to how you would set up devices or enclosures to catch pests that invade your home, website owners set up honeypot traps to catch scrapers. These traps are typically links that are not visible by humans but are visible to web scrapers.

The intention is to get information about the scraper, such as its IP address, so they can block the scraper’s access to the website.

Real-time data scraping

There are certain scenarios where you may need data to be scrapped in real time (for example, price comparisons). As changes can occur anytime, the scraper must constantly monitor the website and scrape data. Acquiring large amounts of data in real time is challenging.

Web scraping best practices

Now that you’re aware of the challenges you may face, it’s important to know the best practices to ensure that you are ethically scraping web data.

Respect robots.txt

One of the challenges that you’ll face when scraping web data is abiding by the terms of robots.txt. It is a best practice to follow the guides set by a website around what a web scrape can and cannot crawl.

If a website does not permit web scraping, it is unethical to scrape that website. It’s better to find another data source or contact the website owner directly and discuss a solution.

Be nice to servers

Web servers can only take so much. Exceeding a web server’s load results in the server crashing. Consider what may be an acceptable frequency of requests to make to a host’s server. Several requests in a short time span may result in server failure, which in turn disrupts the user experience for other visitors to the website.

Maintain a reasonable time lapse between requests and be considerate with the number of parallel requests.

Scrape during off-peak hours

Understand when a website may receive its most traffic and refrain from scraping during these hours. Your goal is not to hamper the user experience of other visitors. It’s your moral responsibility to scrape when a website receives less traffic. (This also works in your favor as it significantly improves the speed of your scraper.)

Scrapy tutorial

Web scraping in Python usually involves coding several menial tasks from scratch. However, Scrapy, an open-source web crawling framework, deals with several of the common start-up requirements by default. This means that you can focus on extracting the data that you need from the target websites.

To demonstrate the power of Scrapy, you develop a spider, which is a Scrapy class where you define the behavior of your web scraper. Use this spider to scrape all the listings from the Boston Realty Advisors website (Figure 1).

Inspecting the target website

Before starting any web scraping project, it is important to inspect the target website to scrape. The first thing that I like to check when examining a website is whether the pages are static or generated by JavaScript. 

To bring up the developer toolbox, press F12. On the Network tab, make sure that Disable cache is checked.

To bring up the command palette, press CTRL + SHIFT + P (Windows Linux) or Command + SHIFT + P (Mac). Type Disable JavaScript, then press Enter and reload the target website.

Figure 2 shows how the Boston Realty Advisors website looks without JavaScript. 

The empty page tells you that the target web page is generated using JavaScript. You can’t scrape the page by trying to parse the HTML elements displayed in the Elements tab of the developer toolbox. Re-enable JavaScript in the developer tools and reload the page.

There’s more to examine.

In the developer toolbox, choose the XHR (XML HTTP Request) tab. After browsing the requests sent to the server, I noticed something interesting. There are two requests called “inventory,” but in one, you have access to all the data on the first page in JSON.

This information is great because it means that you don’t have to visit the main listing page on the Boston Realty Advisors website to get access to the data you want.

Now you are ready to begin scraping. (I did more inspections to better understand how to mimic the requests being made to the server, but that is beyond the scope of this post.)

Creating the Scrapy project

To set up the Scrapy project, first install scrapy. I recommend doing this step in a virtual environment.

pip install scrapy

After the virtual environment is activated, enter the following command:

scrapy startproject bradvisors

This command creates a Scrapy project called bradvisors. Scrapy also automatically adds some files to the directory.

After running the command, the final directory structure looks like the following tree:

.
└── bradvisors
   ├── bradvisors
   │   ├── __init__.py
   │   ├── items.py
   │   ├── middlewares.py
   │   ├── pipelines.py
   │   ├── settings.py
   │   └── spiders
   │       └── __init__.py
   └── scrapy.cfg

So far, you’ve inspected the elements of the website to scrape and created the Scrapy project.

Building the spider

The spider module must be built in the bradvisors/bradvisors/spiders directory. The name of my spider script is bradvisors_spider.py but you can use a custom name.

The following code extracts the data from this website. The code example only runs successfully when the items.py file is updated. For more information, see the explanation after the example.

import json

import scrapy
from bradvisors.items import BradvisorsItem

class BradvisorsSpider(scrapy.Spider):
    name = "bradvisors"
    start_urls = ["https://bradvisors.com/listings/"]
  
    url = "https://buildout.com/plugins/5339d012fdb9c122b1ab2f0ed59a55ac0327fd5f/inventory"

    headers = {
    'authority': 'buildout.com',
    'accept': 'application/json, text/javascript, */*; q=0.01',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
    'cache-control': 'no-cache',
    'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
    'origin': 'https://buildout.com',
    'pragma': 'no-cache',
    'referer': 'https://buildout.com/plugins/5339d012fdb9c122b1ab2f0ed59a55ac0327fd5f/bradvisors.com/inventory/?pluginId=0&iframe=true&embedded=true&cacheSearch=true&=undefined',
    'sec-ch-ua': '"Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"',
    'sec-fetch-dest': 'empty',
    'sec-fetch-mode': 'cors',
    'sec-fetch-site': 'same-origin',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36',
    'x-newrelic-id': 'Vg4GU1RRGwIJUVJUAwY=',
    'x-requested-with': 'XMLHttpRequest'
    }
  
    def parse(self, response):
        url = "https://buildout.com/plugins/5339d012fdb9c122b1ab2f0ed59a55ac0327fd5f/inventory"
        # There are 5 pages on the website
        for i in range(5):
            # Change the page number in the payload
            payload = f"utf8=%E2%9C%93&polygon_geojson=&lat_min=&lat_max=&lng_min=&lng_max=&mobile_lat_min=
        &mobile_lat_max=&mobile_lng_min=&mobile_lng_max=&page={str(i)}&map_display_limit=500&map_type=roadmap
            &custom_map_marker_url=%2F%2Fs3.amazonaws.com%2Fbuildout-production%2Fbrandings%2F7242%2Fprofile_photo
                %2Fsmall.png%3F1607371909&use_marker_clusterer=true&placesAutoComplete=&q%5Btype_use_offset_eq_any%5D%5B%5D=
                    &q%5Bsale_or_lease_eq%5D=&q%5Bbuilding_size_sf_gteq%5D=&q%5Bbuilding_size_sf_lteq%5D=&q%5B
                        listings_data_max_space_available_on_market_gteq%5D=&q%5Blistings_data_min_space_available_on_market_lteq
                            %5D=&q%5Bproperty_research_property_year_built_gteq%5D=&q%5Bproperty_research_property_year_built_lteq
                                %5D=&q%5Bproperty_use_id_eq_any%5D%5B%5D=&q%5Bcompany_office_id_eq_any%5D%5B%5D=&q%5Bs%5D%5B%5D="
            # Crawl the data, given the payload
            yield scrapy.Request(method="POST", body=payload, url=url, headers=self.headers, callback=self.parse_api)

    def parse_api(self, response):
        # Response is json, use loads to convert it into Python dictionary
        data = json.loads(response.body)

        # Our item object defined in items.py
        item = BradvisorsItem()

        for listing in data["inventory"]:
            item["address"] = listing["address_one_line"]
            item["city"] = listing["city"]
            item["city_state"] = listing["city_state"]
            item["zip"] = listing["zip"]
            item["description"] = listing["description"]
            item["size_summary"] = listing["size_summary"]
            item["item_url"] = listing["show_link"]
            item["property_sub_type_name"] = listing["property_sub_type_name"]
            item["sale"] = listing["sale"]
            item["sublease"] = listing["sublease"]
            yield item

The code accomplishes the following tasks:

1. Defines the name of the scraper as bradvisors.
2. Defines the headers to pass with the request.
3. Specifies that the parse method is the automatic call back when the scraper is run.
4. Defines the parse method, where you iterate through the number of pages to scrape, pass the page number to the payload, and yield that request. This calls back the parse_api method on each iteration.
5. Defines the parse_api method and converts the valid JSON response into a Python dictionary.
6. Defines the BradvisorsItem class in items.py (next code example).
7. Loops through all the listings in the inventory and scrapes specific elements.

# items.py
import scrapy

class BradvisorsItem(scrapy.Item):
   # define the fields for your item here like:
   # name = scrapy.Field()
   address = scrapy.Field()
   city = scrapy.Field()
   city_state = scrapy.Field()
   zip = scrapy.Field()
   description = scrapy.Field()
   size_summary = scrapy.Field()
   item_url = scrapy.Field()
   property_sub_type_name = scrapy.Field()
   sale = scrapy.Field()

Next, you must execute the scrape to parse the data.

Running the scraper

Navigate to the project’s root directory from the command line (in this case, that is bradvisors). Run the following command:

scrapy crawl bradvisors -o data.csv

This command scrapes the Boston Realty Advisors website and saves the extracted data in a data.csv file in the project’s root directory.

Great, you have now acquired real estate data!

What’s next?

As the demand for big data grows, equipping yourself with the ability to acquire data with a web scraping tool is an extremely valuable skill set. Scraping data from the Internet can present several challenges. As well as acquiring the data that you require, your goal should be to treat websites respectfully and scrape them ethically. 

Did you find this Scrapy tutorial helpful? Leave your feedback in the comments or connect with me:

• /kurtispykes on LinkedIn
• @KurtisPykes on Twitter
• /kurtispykes on GitHub

Web scraping FAQ

Is web scraping illegal?

No, web scraping is legal, given that the data you are scraping is public. Search engines such as Google scrape web data daily to curate search results for their users.

Is web scraping free?

You can pay for web scraping services to simplify the web scraping process. You could also learn a programming language and do it yourself for free.

What tools can I use for web scraping in Python?

There are several tools for web scraping in Python, such as Beautiful Soup, MechanicalSoup, Requests (Python module), Scrapy, Selenium, and Urllib.

A smart hospital relies on data-driven insights, including machine learning models and AI-powered medical devices, to facilitate decision-making.

A smart hospital relies on data-driven insights, including machine learning models and AI-powered medical devices, to facilitate decision-making.