NVIDIA XGBoost 3.0: Training Terabyte-Scale Datasets with Grace Hopper Superchip

NVIDIA XGBoost 3.0: Training Terabyte-Scale Datasets with Grace Hopper Superchip

 

NVIDIA has unveiled a major milestone in scalable machine learning: XGBoost 3.0, now able to train gradient-boosted decision tree (GBDT) models from gigabytes up to 1 terabyte (TB) on a single GH200 Grace Hopper Superchip. The breakthrough enables companies to process immense datasets for applications like fraud detection, credit risk modeling, and algorithmic trading, simplifying the once-complex process of scaling machine learning ML pipelines.

Breaking Terabyte Barriers

At the heart of this advancement is the new External-Memory Quantile DMatrix in XGBoost 3.0. Traditionally, GPU training was limited by the available GPU memory, capping achievable dataset size or forcing teams to adapt complex multi-node frameworks. The new release leverages the Grace Hopper Superchip’s coherent memory architecture and ultrafast 900GB/s NVLink-C2C bandwidth. This enables direct streaming of pre-binned, compressed data from host RAM into the GPU, overcoming bottlenecks and memory constraints that previously required RAM-monster servers or large GPU clusters.

Real-World Gains: Speed, Simplicity, and Cost Savings

Institutions like the Royal Bank of Canada (RBC) have reported up to 16x speed boosts and a 94% reduction in total cost of ownership (TCO) for model training by moving their predictive analytics pipelines to GPU-powered XGBoost. This leap in efficiency is crucial for workflows with constant model tuning and rapidly changing data volumes, allowing banks and enterprises to optimize features faster and scale as data grows.

How It Works: External Memory Meets XGBoost

The new external-memory approach introduces several innovations:

  • External-Memory Quantile DMatrix: Pre-bins every feature into quantile buckets, keeps data compressed in host RAM, and streams it as needed, maintaining accuracy while reducing GPU memory load.
  • Scalability on a Single Chip: One GH200 Superchip, with 80GB HBM3 GPU RAM plus 480GB LPDDR5X system RAM, can now handle a full TB-scale dataset—tasks formerly possible only across multi-GPU clusters.
  • Simpler Integration: For data science teams using RAPIDS, activating the new method is a straightforward drop-in, requiring minimal code changes.

Technical Best Practices

  • Use grow_policy='depthwise' for tree construction for best performance on external memory.
  • Run with CUDA 12.8+ and an HMM-enabled driver for full Grace Hopper support.
  • Data shape matters: the number of rows (labels) is the main limiter for scaling—wider or taller tables yield comparable performance on the GPU.

Upgrades

Other highlights in XGBoost 3.0 include:

  • Experimental support for distributed external memory across GPU clusters.
  • Reduced memory requirements and initialization time, notably for mostly-dense data.
  • Support for categorical features, quantile regression, and SHAP explainability in external-memory mode.

Industry Impact

By bringing terabyte-scale GBDT training to a single chip, NVIDIA democratizes access to massive machine learning for both financial and enterprise users, paving the way for faster iteration, lower cost, and lower IT complexity.

XGBoost 3.0 and the Grace Hopper Superchip together mark a major leap forward in scalable, accelerated machine learning.


Check out the Technical details. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

The post NVIDIA XGBoost 3.0: Training Terabyte-Scale Datasets with Grace Hopper Superchip appeared first on MarkTechPost.

MarkTechPost

Read More
Meet CoAct-1: A Novel Multi-Agent System that Synergistically Combines GUI-based Control with Direct Programmatic Execution

Meet CoAct-1: A Novel Multi-Agent System that Synergistically Combines GUI-based Control with Direct Programmatic Execution

 

A Team of researchers from USC, Salesforce AI and University of Washington have introduced CoAct-1, a pioneering multi-agent computer-using agent (CUA) that marks a significant leap in autonomous computer operation. By elevating coding to a first-class action—on par with traditional GUI manipulation—CoAct-1 overcomes longstanding challenges of efficiency and reliability in complex, long-horizon computer tasks. On the demanding OSWorld benchmark, CoAct-1 sets a new gold standard, achieving a state-of-the-art (SOTA) success rate of 60.76%, making it the first CUA agent to surpass the 60% mark.

Why CoAct-1? Bridging the Efficiency Gap in Computer-Using Agents

Conventional CUA agents rely solely on pixel-based GUI interaction—emulating human users by clicking, typing, and navigating interfaces. While this approach mimics user workflows, it proves fragile and inefficient for intricate, multi-step tasks, especially those involving dense UI layouts, multi-app pipelines, or complex OS operations. Single errors such as a mis-click can derail entire workflows, and sequence lengths balloon as tasks increase in complexity.

Efforts to mitigate these issues have included augmenting GUI agents with high-level planners, as seen in systems like GTA-1 and modular multi-agent frameworks. However, these methods cannot escape the bottleneck of GUI-centric action spaces, ultimately limiting both efficiency and robustness.

CoAct-1: Hybrid Architecture with Coding as Action

CoAct-1 takes a fundamentally different approach by integrating three specialized agents:

  • Orchestrator: The high-level planner that decomposes complex tasks and dynamically delegates each subtask either to the Programmer or the GUI Operator based on task requirements.
  • Programmer: Executes backend operations—file management, data processing, environment configuration—directly via Python or Bash scripts, bypassing cumbersome GUI action sequences.
  • GUI Operator: Uses a vision-language model to interact with visual interfaces when human-like UI navigation is indispensable.

This hybrid model enables CoAct-1 to strategically substitute brittle and lengthy mouse-keyboard operations with concise, reliable code execution, while still leveraging GUI interactions where necessary.

Evaluation on OSWorld: Record-Setting Performance

OSWorld—a leading benchmark featuring 369 tasks spanning office productivity, IDEs, browsers, file managers, and multi-app workflows—proves an exacting testbed for agentic systems. Each task mirrors real-world language goals and is assessed by a granular rule-based scoring system.

Results

  • Overall SOTA Success Rate: CoAct-1 achieves 60.76% on the 100+ step category—the first CUA agent to cross the 60-point threshold. This outpaces GTA-1 (53.10%), OpenAI CUA 4o (31.40%), UI-TARS-1.5 (29.60%), and other leading frameworks.
  • Stepped Allowance Performance: At a 100-step budget, CoAct-1 scores 59.93%, again leading all competitors.
  • Efficiency: Completes tasks with an average of 10.15 steps per successful task, compared to 15.22 for GTA-1, 14.90 for UI-TARS, and with much higher success than OpenAI CUA 4o, which, despite fewer steps (6.14), achieves only 31.40% success.

Breakdown

CoAct-1 dominates across task types, with especially large gains in workflows benefitting from code execution:

  • Multi-App: 47.88% (vs. GTA-1’s 38.34%)
  • OS Tasks: 75.00%
  • VLC: 66.07%
  • In productivity and IDE domains (LibreOffice Calc, Writer, VSCode), it consistently leads or ties with the SOTA.

Key Insights: What Drives CoAct-1’s Gains?

  • Coding Actions Replace Redundant GUI Sequences: For operations like batch image resizing or advanced file manipulations, single scripts replace dozens of error-prone clicks, reducing both steps and risk of failure.
  • Dynamic Delegation: The Orchestrator’s flexible task assignment ensures optimal use of coding vs. GUI actions.
  • Improvement with Stronger Backbones: The best configuration uses OpenAI CUA 4o for the GUI Operator, OpenAI o3 for the Orchestrator, and o4-mini for the Programmer, reaching the top 60.76% score. Systems using only smaller or less capable backbones score significantly lower.
  • Efficiency Correlates with Reliability: Fewer steps directly reduce opportunities for error—the single strongest predictor of successful completion.

Conclusion: A Leap Forward in Generalized Computer Automation

By making coding a first-class system action alongside GUI manipulation, CoAct-1 delivers both a quantum leap in success and efficiency, and illustrates the practical path forward for scalable, reliable autonomous computer agents. Its hybrid architecture and dynamic execution logic set a new high-water mark for the CUA field, heralding robust advances in real-world computer automation.


Check out the Paper and Technical details. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

The post Meet CoAct-1: A Novel Multi-Agent System that Synergistically Combines GUI-based Control with Direct Programmatic Execution appeared first on MarkTechPost.

MarkTechPost

Read More
Proxy Servers Explained: Types, Use Cases & Trends in 2025 [Technical Deep Dive]

Proxy Servers Explained: Types, Use Cases & Trends in 2025 [Technical Deep Dive]

 

Estimated reading time: 5 minutes

Introduction

A proxy server is a vital intermediary between clients and destination servers, facilitating both security and speed in the modern internet. In 2025, with digital privacy, enterprise security, and data-driven automation to the forefront, proxy servers are indispensable for individuals and organizations. The global web proxy market is projected to reach $50 billion by 2026, propelled by ongoing privacy and compliance demands.

What Is a Proxy Server?

A proxy server is a dedicated system or software relay that takes requests from a client (e.g., a browser) and forwards them to a target server. The proxy then collects the server’s response and returns it to the client. This arrangement can filter, cache, monitor, and secure traffic flows.

Technical Architecture:

Key Functions (2025):

  • Request/response interception and filtering
  • Protocol translation (HTTP, HTTPS, SOCKS5)
  • Web content caching for speed
  • Traffic monitoring and logging
  • IP address masking for privacy
  • Encrypted connections (TLS 1.3, Encrypted SNI)

Types of Proxy Servers

Proxy Type Hides IP Reveals Proxy Client Config Common Use Case
Forward Yes Yes Required Anonymous browsing, filtering
Reverse No No Server-side Load balancing, WAF, DDoS defense
Transparent No No Not needed Caching, monitoring
Anonymous Yes Yes Required Geo content access, basic privacy
Elite (High-Anon) Yes No Required Secure scraping, maximum privacy

Descriptions:

  • Forward Proxy: Works on behalf of the client, enabling privacy, content filtering, and access control.
  • Reverse Proxy: Handles traffic before it reaches backend servers, enabling load balancing, web security, and SSL termination.
  • Transparent Proxy: Operates without client knowledge, mainly for caching and policy enforcement by ISPs or organizations.
  • Anonymous Proxy: Hides real IP address but reveals presence of a proxy, useful for regional content access.
  • High-Anonymity Proxy: Masks both IP and proxy use, crucial for advanced privacy and large-scale automation.

Key Use Cases in 2025

  • Online Privacy & Anonymity: Essential for journalists, activists, and privacy seekers.
  • Content Filtering: Enterprises block malicious or unwanted websites.
  • Network Security & Firewalls: Reverse proxies enforce security, inspect traffic, and shield against attacks.
  • Load Balancing & Scalability: Distributes traffic for uptime and performance.
  • Web Caching: Speeds up delivery of popular content and reduces server load.
  • AI Automation & Scraping: Proxies enable distributed, stealthy data collection for business intelligence.

Emerging Trends in Proxy Servers (2025)

  • AI-Driven Traffic Analysis: Proxies now integrate machine learning for anomaly detection and route optimization.
  • Edge Proxy Servers: Positioned near users for lower latency, vital for CDNs and IoT.
  • Zero Trust Security: Proxies act as policy enforcers at every network access point.
  • Encrypted Communication: TLS 1.3 and Encrypted SNI protect proxy traffic from surveillance.
  • Performance Optimization: Advanced connection pooling and protocol upgrades (HTTP/2, SOCKS5) deliver up to 80% latency improvements.

Top Proxy Server Providers in 2025

The following companies lead the market with robust feature sets, massive global coverage, high reliability, and strong user feedback:

Provider IP Pool Size / Coverage Proxy Types Key Strengths / Notes Starting Price
Oxylabs 177M+ IPs, 195+ countries Residential, Datacenter, ISP, Mobile, Web Unblocker Enterprise-scale, powerful geo-targeting, great support $4/GB residential, $12/mo datacenter
Bright Data 150M+ residential IPs, 195+ countries Residential, Datacenter, ISP, Mobile Largest pool, precise geo-targeting, strong compliance $2.94/GB residential
Decodo (Smartproxy) 125M+ IPs, 195+ countries Residential, ISP, Datacenter, Mobile Value leader, user-friendly, fast support $1.5/GB residential, $5.55/mo datacenter
SOAX 191M+ IPs, global reach Residential, Mobile, ISP Low-latency, ethical sourcing, premium SOCKS5 support $4/GB, $90/port/mo
Webshare 30M+ IPs, 195+ countries Datacenter, Residential, Static Developer-focused, affordable, easy setup $2.99/mo datacenter, free trial
IPRoyal 32M+ IPs, 195+ countries Residential, Datacenter, ISP, Mobile Rotating/sticky IPs, budget-friendly, SOCKS5 support $2.45/GB residential
Rayobyte 300K datacenter, 40M+ residential Datacenter, Residential, ISP, Mobile US coverage, transparent geo-targeting $3.50/GB residential, datacenter varies

Provider Notes:

  • Leading providers: Oxylabs and Bright Data set industry benchmarks for reliability and sheer IP numbers.
  • Budget options: Decodo (Smartproxy), IPRoyal, and Webshare offer affordable plans and flexible trials.
  • Premium features: SOAX provides ethical pools, low latency, and advanced protocol support.
  • User experience: Most vendors offer free trial periods and expert support in multiple languages.

Choosing a Provider:

Consider IP pool size, coverage, protocol support (HTTP/HTTPS/SOCKS5), platform tools, customer support, compliance, and pricing structure.

Conclusion

In 2025, proxy servers are fundamental to secure, private, and optimized internet usage—from web privacy to network defense and scalable automation. New advances in AI, encryption, and edge networks are transforming proxy technology into smarter, faster, and more resilient tools.
Selecting the right proxy type and provider is crucial for architects, sys-admins, businesses, and individuals seeking robust connectivity and compliance amid evolving digital challenges.


FAQ: Frequently Asked Questions about Proxy Servers

  • What is a proxy server used for? A proxy server is used for anonymizing internet traffic, content filtering, web caching, improving security, and balancing network loads.
  • How does a proxy server improve privacy? It hides the user’s real IP address by forwarding requests through an intermediary server, thus masking identity from destination sites.
  • What’s the difference between a forward and reverse proxy? A forward proxy sits in front of the client and filters outbound requests, while a reverse proxy sits in front of the server and manages inbound traffic.
  • Are proxy servers legal? Yes, using a proxy server is legal in most countries. However, using proxies for illegal activities (e.g., fraud, hacking) is not.
  • What is the safest type of proxy server? High-anonymity or elite proxies offer the most privacy and are typically considered the safest for sensitive use cases.
  • Can I set up my own proxy server? Yes, you can set up your own proxy using tools like Squid, HAProxy, or Nginx, either on a local machine or cloud instance.

The post Proxy Servers Explained: Types, Use Cases & Trends in 2025 [Technical Deep Dive] appeared first on MarkTechPost.

MarkTechPost

Read More
What is a Proxy Server? A Technical Deep Dive with Trends and Top Proxy Servers (2025 Edition)

What is a Proxy Server? A Technical Deep Dive with Trends and Top Proxy Servers (2025 Edition)

 

Estimated reading time: 4 minutes

Introduction

A proxy server is a vital intermediary between clients and destination servers, facilitating both security and speed in the modern internet. In 2025, with digital privacy, enterprise security, and data-driven automation to the forefront, proxy servers are indispensable for individuals and organizations. The global web proxy market is projected to reach $50 billion by 2026, propelled by ongoing privacy and compliance demands.

What Is a Proxy Server?

A proxy server is a dedicated system or software relay that takes requests from a client (e.g., a browser) and forwards them to a target server. The proxy then collects the server’s response and returns it to the client. This arrangement can filter, cache, monitor, and secure traffic flows.

Technical Architecture:

Key Functions (2025):

  • Request/response interception and filtering
  • Protocol translation (HTTP, HTTPS, SOCKS5)
  • Web content caching for speed
  • Traffic monitoring and logging
  • IP address masking for privacy
  • Encrypted connections (TLS 1.3, Encrypted SNI)

Types of Proxy Servers

Proxy Type Hides IP Reveals Proxy Client Config Common Use Case
Forward Yes Yes Required Anonymous browsing, filtering
Reverse No No Server-side Load balancing, WAF, DDoS defense
Transparent No No Not needed Caching, monitoring
Anonymous Yes Yes Required Geo content access, basic privacy
Elite (High-Anon) Yes No Required Secure scraping, maximum privacy

Descriptions:

  • Forward Proxy: Works on behalf of the client, enabling privacy, content filtering, and access control.
  • Reverse Proxy: Handles traffic before it reaches backend servers, enabling load balancing, web security, and SSL termination.
  • Transparent Proxy: Operates without client knowledge, mainly for caching and policy enforcement by ISPs or organizations.
  • Anonymous Proxy: Hides real IP address but reveals presence of a proxy, useful for regional content access.
  • High-Anonymity Proxy: Masks both IP and proxy use, crucial for advanced privacy and large-scale automation.

Key Use Cases in 2025

  • Online Privacy & Anonymity: Essential for journalists, activists, and privacy seekers.
  • Content Filtering: Enterprises block malicious or unwanted websites.
  • Network Security & Firewalls: Reverse proxies enforce security, inspect traffic, and shield against attacks.
  • Load Balancing & Scalability: Distributes traffic for uptime and performance.
  • Web Caching: Speeds up delivery of popular content and reduces server load.
  • AI Automation & Scraping: Proxies enable distributed, stealthy data collection for business intelligence.

Emerging Trends in Proxy Servers (2025)

  • AI-Driven Traffic Analysis: Proxies now integrate machine learning for anomaly detection and route optimization.
  • Edge Proxy Servers: Positioned near users for lower latency, vital for CDNs and IoT.
  • Zero Trust Security: Proxies act as policy enforcers at every network access point.
  • Encrypted Communication: TLS 1.3 and Encrypted SNI protect proxy traffic from surveillance.
  • Performance Optimization: Advanced connection pooling and protocol upgrades (HTTP/2, SOCKS5) deliver up to 80% latency improvements.

Top Proxy Server Providers in 2025

The following companies lead the market with robust feature sets, massive global coverage, high reliability, and strong user feedback:

Provider IP Pool Size / Coverage Proxy Types Key Strengths / Notes Starting Price
Oxylabs 177M+ IPs, 195+ countries Residential, Datacenter, ISP, Mobile, Web Unblocker Enterprise-scale, powerful geo-targeting, great support $4/GB residential, $12/mo datacenter
Bright Data 150M+ residential IPs, 195+ countries Residential, Datacenter, ISP, Mobile Largest pool, precise geo-targeting, strong compliance $2.94/GB residential
Decodo (Smartproxy) 125M+ IPs, 195+ countries Residential, ISP, Datacenter, Mobile Value leader, user-friendly, fast support $1.5/GB residential, $5.55/mo datacenter
SOAX 191M+ IPs, global reach Residential, Mobile, ISP Low-latency, ethical sourcing, premium SOCKS5 support $4/GB, $90/port/mo
Webshare 30M+ IPs, 195+ countries Datacenter, Residential, Static Developer-focused, affordable, easy setup $2.99/mo datacenter, free trial
IPRoyal 32M+ IPs, 195+ countries Residential, Datacenter, ISP, Mobile Rotating/sticky IPs, budget-friendly, SOCKS5 support $2.45/GB residential
Rayobyte 300K datacenter, 40M+ residential Datacenter, Residential, ISP, Mobile US coverage, transparent geo-targeting $3.50/GB residential, datacenter varies

Provider Notes:

  • Leading providers: Oxylabs and Bright Data set industry benchmarks for reliability and sheer IP numbers.
  • Budget options: Decodo (Smartproxy), IPRoyal, and Webshare offer affordable plans and flexible trials.
  • Premium features: SOAX provides ethical pools, low latency, and advanced protocol support.
  • User experience: Most vendors offer free trial periods and expert support in multiple languages.

Choosing a Provider:

Consider IP pool size, coverage, protocol support (HTTP/HTTPS/SOCKS5), platform tools, customer support, compliance, and pricing structure.

Conclusion

In 2025, proxy servers are fundamental to secure, private, and optimized internet usage—from web privacy to network defense and scalable automation. New advances in AI, encryption, and edge networks are transforming proxy technology into smarter, faster, and more resilient tools.
Selecting the right proxy type and provider is crucial for architects, sys-admins, businesses, and individuals seeking robust connectivity and compliance amid evolving digital challenges.

The post What is a Proxy Server? A Technical Deep Dive with Trends and Top Proxy Servers (2025 Edition) appeared first on MarkTechPost.

MarkTechPost

Read More
Alibaba Introduces Group Sequence Policy Optimization (GSPO): An Efficient Reinforcement Learning Algorithm that Powers the Qwen3 Models

Alibaba Introduces Group Sequence Policy Optimization (GSPO): An Efficient Reinforcement Learning Algorithm that Powers the Qwen3 Models

 

Reinforcement learning (RL) plays a crucial role in scaling language models, enabling them to solve complex tasks such as competition-level mathematics and programming through deeper reasoning. However, achieving stable and reliable training dynamics is a challenge when scaling RL with larger computational resources. Current state-of-the-art algorithms, such as GRPO, struggle with serious stability issues during the training of gigantic language models, often resulting in catastrophic failures. These instabilities arise from incorrect use of importance sampling weight applications, which introduce high-variance noise. This noise accumulates with longer responses and is worsened by clipping mechanisms. This causes model collapse and hinders progress.

Existing methods like PPO and GRPO rely on mechanisms like clipping to address off-policy learning challenges where responses are taken from outdated policies. However, these approaches face limitations due to their ill-posed objectives, particularly in large models handling long-response tasks. GRPO’s token-level importance sampling introduces high-variance noise and irreversible model collapse. Attempts to recover from collapse through hyperparameter tuning or checkpoint restoration fail, highlighting a fundamental design flaw. The mismatch between token-level corrections and sequence-level rewards emphasizes the need for a new approach that optimizes directly at the sequence level to ensure stability and scalability.

Researchers from Alibaba Inc. have proposed Group Sequence Policy Optimization (GSPO), an RL algorithm designed to train LLMs. GSPO’s main innovation lies in its theoretically grounded importance ratio, derived from sequence likelihood, which aligns with the principles of importance sampling. Moreover, it calculates normalized rewards as advantages for multiple responses to a query, promoting consistency between sequence-level rewards and optimization goals. Empirical evaluations reveal that GSPO significantly outperforms GRPO in stability, efficiency, and overall performance. By resolving stability challenges in training large Mixture-of-Experts (MoE) models, GSPO eliminates the need for complex stabilization techniques.

Researchers use a cold-start model fine-tuned from Qwen3-30B-A3B-Base for the experiment, reporting the training reward curves and the model performance curves across AIME’24, LiveCodeBench, and CodeForces benchmarks. During training, rollout data in each batch is split into four mini-batches for gradient updates. GSPO clips entire responses rather than individual tokens, with clipping ranges set to 3e-4 and 4e-4 in its formulation. This leads to a two-order-of-magnitude difference in clipped token fractions compared to GRPO. Despite removing more tokens for gradient estimation, GSPO achieves higher training efficiency. This result highlights the inefficiency of GRPO’s noisy token-level estimates.

GSPO offers significant advantages for MoE training by stabilizing the process through consistent expert activations across gradient updates, unlike GRPO, which struggles with expert-activation volatility. This removes the need for complex solutions like Routing Replay, simplifying the infrastructure and allowing models to utilize their full capacity. In RL infrastructure, GSPO’s sequence-level optimization reduces dependency on token-level likelihoods, making it more robust to precision mismatch. This enables direct use of inference engine likelihoods, avoiding costly recomputation and improving efficiency in partial rollouts and multi-turn RL. GSPO also streamlines RL infrastructure for large-scale language model training.

In conclusion, researchers introduced Group Sequence Policy Optimization (GSPO), an RL algorithm designed for training LLMs. GSPO builds on the principles of importance sampling and introduces sequence-level clipping, rewarding, and optimization to overcome the instability and inefficiency seen in GRPO. Its superior performance in training stability, efficiency, and scalability, particularly for MoE models, emphasizes its importance as a strong algorithmic foundation. The advancements made possible by GSPO have played a key role in the remarkable performance of the Qwen3 models. Building on GSPO as a foundational approach, researchers plan to expand RL methods, opening the door for groundbreaking progress in AI.


Check out the Paper. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

The post Alibaba Introduces Group Sequence Policy Optimization (GSPO): An Efficient Reinforcement Learning Algorithm that Powers the Qwen3 Models appeared first on MarkTechPost.

MarkTechPost

Read More
Google AI Releases DeepPolisher: A New Deep Learning Tool that Improves the Accuracy of Genome Assemblies by Precisely Correcting Base-Level Errors

Google AI Releases DeepPolisher: A New Deep Learning Tool that Improves the Accuracy of Genome Assemblies by Precisely Correcting Base-Level Errors

 

Google AI, in collaboration with the UC Santa Cruz Genomics Institute, has introduced DeepPolisher, a cutting-edge deep learning tool designed to substantially improve the accuracy of genome assemblies by correcting base-level errors. Its notable efficacy was recently demonstrated in advancing the Human Pangenome Reference, a major milestone in genomics research.

The Challenge of Accurate Genome Assembly

A reference genome is an essential foundation for understanding genetic diversity, heredity, disease mechanisms, and evolutionary biology. Modern sequencing technologies, including those developed by Illumina and Pacific Biosciences, have dramatically improved sequencing accuracy and throughput—but even with technological breakthroughs, assembling an error-free human genome (comprising over 3 billion nucleotides) remains immensely challenging. Even a minuscule per-base error rate can result in thousands of errors which can obscure key genetic variations or mislead downstream analyses.

What Is DeepPolisher?

DeepPolisher is an open-source, transformer-based sequencing correction tool. Building on advances from DeepConsensus, it takes advantage of transformer deep learning architectures to further reduce errors in genome assembly, particularly insertion and deletion (indel) errors, which have a profound impact by shifting reading frames and can cause important genes or regulatory elements to be missed during annotation.

  • Technology: Encoder-only transformer, adapting proven techniques in natural language processing for genomics.
  • Training data: Leveraged a human cell line extensively characterized by NIST and NHGRI, sequenced with various platforms to ensure near-complete accuracy (~99.99999% correctness, between 300–1,000 errors in 6 billion bases).

How Does It Work? (Technical Overview)

  1. Input Alignment: Takes aligned PacBio HiFi reads against a haplotype-resolved genome assembly as input.
  2. Error Site Detection: Scans the assembly in 25kb windows; identifies candidate error sites where read evidence deviates from the assembly.
  3. Data Encoding: For each window containing putative errors (<100bp), it creates a multi-channel tensor representation of read alignment features such as base, base quality, mapping quality, and match/mismatch status.
  4. Model Inference: Feeds these tensors into the transformer, which predicts corrected sequences for these regions.
  5. Output Correction: Outputs differences in VCF format, which are then applied to the assembly to produce a polished, highly accurate sequence using tools like bcftools.

Performance and Impact

DeepPolisher delivers substantial improvements:

  • Total error reduction: ~50%
  • Indel error reduction: >70%
  • Error rates: Achieves an error rate as low as one base error per 500,000 assembled bases in real-world deployment with the Human Pangenome Reference Consortium (HPRC).
  • Genomic Q-score improvement: Raises assembly quality from Q66.7 to Q70.1 on average (Q-score is a logarithmic measure of per-base error rate; higher is better. Q70.1 implies <1 error per 12 million nucleotides)
  • Every sample tested by HPRC showed improvement.

These advances directly impact the reliability and accuracy of derived references, such as in the Human Pangenome Reference, which saw a fivefold data expansion and substantial error reduction due to DeepPolisher.

Deployment and Applications

  • Integrated in major projects: Used in HPRC’s second data release, providing high-accuracy reference assemblies for 232 individuals, ensuring broad ancestral diversity in genomic references.
  • Open-source access: Available via GitHub, with case studies and Dockerized workflows for use on assemblies produced by tools like HiFiasm and sequenced with PacBio HiFi reads.
  • Generalizability: While initially focused on human genomes, the structure and approach are adaptable to other organisms and sequencing platforms, fostering accuracy across the genomics community.

Practical Workflow Example

A typical workflow using DeepPolisher might involve:

  • Input: HiFiasm diploid assembly and PacBio HiFi reads, phase-aligned using the PHARAOH pipeline.
  • Running: Dockerized commands for image creation, inference, and correction application.
  • Output: Separate VCF files for maternal and paternal assemblies, polished FASTAs after bcftools consensus step.
  • Assessment: Use of benchmarking tools (e.g., dipcall, Hap.py) to quantify improvements in error rates and variant accuracy.

Conclusion and Future Directions

DeepPolisher represents a leap forward in genome polishing technology—sharply reducing error rates and unlocking higher resolution for functional genomics, rare variant discovery, and clinical applications. By targeting the remaining barrier to perfect genome assemblages, it enables more accurate diagnosis, population-level genetic studies, and paves the way for next-generation reference projects benefiting biomedical research and medicine.


Check out the Technical details, GitHub Page and Paper. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

The post Google AI Releases DeepPolisher: A New Deep Learning Tool that Improves the Accuracy of Genome Assemblies by Precisely Correcting Base-Level Errors appeared first on MarkTechPost.

MarkTechPost

Read More
OpenAI Just Released GPT-5: The Smartest, Fastest, and Most Useful OpenAI Model

OpenAI Just Released GPT-5: The Smartest, Fastest, and Most Useful OpenAI Model

 

OpenAI just released GPT-5, marking a substantial leap in generative AI, introducing advanced capabilities that cater to both general and highly specialized tasks. This article provides a deep technical dive into GPT-5’s architecture, new features, performance improvements, and the strategic implications for developers, enterprises, and the AI ecosystem.

Architectural Advancements and System Design

GPT-5 is described as OpenAI’s smartest, fastest, and most useful model yet, supporting “thinking built in” for more robust reasoning. Although explicit parameter counts and training data sizes remain undisclosed, OpenAI positions GPT-5 as smarter across “math, science, finance, law, and more,” indicating further scaling of both the underlying neural architecture and the breadth of its training data.

Key model improvements:

  • Deeper Cognitive Abilities: GPT-5 is engineered for deeper, context-aware reasoning, enabling it to handle highly complex, multi-step problems and “think deeply when you need it to.”
  • Reduced Hallucinations: Enhanced truthfulness and reliability make GPT-5 “less prone to hallucinations and pretending to know things,” a persistent challenge in large language models.
  • Agentic & Coding Proficiency: GPT-5 now supports agentic workflows with improved end-to-end coding skills, more usable code, better design output, and augmented debugging capabilities. It can generate full front-end UIs from minimal input and execute long chains of tool calls autonomously.

Model Features, API Changes, and Steerability

Key API and usability features:

Feature Technical Insight
Minimal Reasoning New ability to minimize output reasoning chains, optimizing for speed or verbosity as needed
Verbosity Parameter Users can now dynamically adjust response length/density in the API for context-appropriate communication
Personality & Interface Choice End-users can customize chatbot “personality” and color, enabling brand-aligned or individual experiences.
Advanced Voice Mode Improved speech understanding and user-specific speaking style adaptation.
Study Mode Provides step-by-step, personalized instructional help in complex learning scenarios.
Deep App Integration GPT-5 supports connecting to Gmail, Google Calendar, Google Drive, SharePoint, and more, leveraging these data sources contextually while maintaining enterprise permission structures.

Enterprise and Developer Impact

For Enterprise:

  • Deployment Models: GPT-5 is now available in ChatGPT Team and will roll out to ChatGPT Enterprise and Edu on August 14, 2025.
  • Reliability for Workflow Automation: OpenAI claims GPT-5 can handle “important work with higher confidence,” supporting critical business functions (writing, research, analysis, coding, problem-solving) with improved accuracy.
  • File & App Contextualization: The model can access and utilize company files and organizational apps securely, enhancing productivity workflows.

For Developers:

  • Better Agentic Task Chaining: GPT-5 excels at autonomous execution of complex toolchains, which is critical for building multi-step agentic applications—an area of intense focus in the latest LLM research.
  • API Flexibility: New “minimal” and “verbosity” features give unprecedented control for workflow integration and app-end user customization.
  • Debugging and UI Generation: Out-of-the-box, GPT-5 produces higher fidelity code and UIs with minimal prompts, shrinking the iterative design and deployment cycles.

Comparative Performance Gains

While OpenAI does not publish benchmark scores yet, the qualitative claims suggest:

  • Dramatic gains in accuracy and hallucination reduction over GPT-4 and GPT-4 Turbo.
  • Notably higher proactive health reasoning and law/science domain expertise, positioning GPT-5 as more than a general-purpose language model but a domain-specific co-pilot for professionals.

Strategic and Ecosystem Implications

Research and Open Questions:

  • Emergent Reasoning: With the introduction of “minimal reasoning” and control over output verbosity, GPT-5 likely demonstrates new emergent capabilities—such as situational awareness of when minimal, maximal, or just-right reasoning is optimal.
  • Agentic Use Cases: The model’s ability to autonomously execute chained tool calls hints at integrated agent-pipeline readiness, supporting workflows such as autonomous software engineering and business process automation.
  • End-User Customization: Personalities, voice, and study modes indicate increased focus on AI-human co-adaptation, making the product more accessible and productive for a diverse global user base.

Release and Availability

  • Released: August 2025
  • Available in: ChatGPT Team now; ChatGPT Enterprise and Edu on August 14, 2025.
  • Integration Readiness: Deep compatibility with Google Workspace, Microsoft SharePoint, and other productivity ecosystems.

Conclusion:

GPT-5 redefines the boundary of language models with advanced reasoning, agentic capabilities, and deep integration features. Its focus on reliability, domain intelligence, personalization, and workflow automation positions it as a foundational model for the next era of general and vertical AI solutions.


Check out the Technical details and Try it here. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter.

The post OpenAI Just Released GPT-5: The Smartest, Fastest, and Most Useful OpenAI Model appeared first on MarkTechPost.

MarkTechPost

Read More
OpenAI’s GPT-5 leaks before official launch

OpenAI’s GPT-5 leaks before official launch

OpenAI's GPT-5 leaks before official launch

In a surprising turn of events, details about OpenAI’s highly anticipated next-generation artificial intelligence (AI) model, GPT-5, have surfaced online ahead of its official launch. Minutes after developers spotted an unlisted repository on GitHub Models, screenshots of “GPT-5” began ricocheting across X and Reddit, forcing OpenAI’s best-kept secret into the public eye a day early.

A now-deleted blog post on GitHub, the Microsoft-owned platform for developers, prematurely announced the arrival of GPT-5, giving the world a sneak peek into what the company is calling its “most advanced model” to date. Although the leak was quickly taken down, a screenshot reveals a suite of next-generation models that promise sharper reasoning, faster code generation, and new agent-style autonomy.

While OpenAI has not confirmed the authenticity, multiple tech sites captured the details before they vanished, lending the leak unusual credibility. The premature announcement, though quickly taken down, was archived and widely shared. The leak confirms that GPT-5 is not a single, monolithic model, but a family of capable AI models with different capabilities, showing OpenAI’s intention to serve a large user base, from individuals and small businesses to large enterprises.

What landed on GitHub?

According to the archived post, GPT-5 will be available in four variants:

  • gpt-5: The flagship model, designed for complex logic and multi-step reasoning tasks.
  • gpt-5-mini: A more lightweight and cost-effective version with solid reasoning for less demanding everyday applications.
  • gpt-5-nano: Optimized for speed and low-latency, ideal for real-time applications.
  • gpt-5-chat: a multimodal, context-aware conversational model built for enterprise assistants.
OpenAI's GPT-5 leaks before official launch

The leaked information shows “major improvements in reasoning, code quality, and user experience.” GPT-5 is said to handle complex coding tasks with minimal prompting and possesses “enhanced agentic capabilities,” making it a more powerful and intelligent assistant.

Flexible Access and Pricing:

According to other leaks, there are now tiered options for ChatGPT users:

  • Free: Basic access to GPT-5, good enough for most day-to-day needs.
  • Plus: Enhanced with advanced reasoning for more demanding scenarios.
  • Pro: Raises the bar on performance, offering the most computation for the hardest challenges and ensuring the best possible answers.
OpenAI's GPT-5 leaks before official launch

The accidental disclosure on GitHub may have stolen some of the thunder from OpenAI’s official announcement, but it has also ignited a wave of excitement and speculation. It also shows that a one-size-fits-all approach is no longer sufficient.OpenAI can easily expand its reach and solidify its position as a leader in artificial intelligence (AI) by offering a spectrum of models.

Broader context

The timing is awkward for OpenAI, which is scheduled to stream a one-hour product event later today. Company watchers had expected incremental updates; however, the GPT-5 is a full generational jump with agentic skills that could outshine rivals like Anthropic’s Claude Opus 4.1.

If the specs hold, GPT -5’s expanded context and native memory could change how businesses automate research, write software, and even run day-to-day knowledge work. The smaller “mini” and “nano” tiers can also push into capable yet cost-sensitive and edge scenarios where GPT-4 still felt too heavy.

In Conclusion:

Leaks are rarely the whole story, but they set expectations. GitHub’s brief cameo of GPT-5 hints at a model family that scales from free chats to research-grade reasoning, all tied into a pricing structure designed to pull users up the subscription ladder. Whether today’s livestream confirms or corrects these details, OpenAI now has to address a simple question: can the official reveal top the surprise the internet has already seen?


🤝
For Partnership/Promotion on AI Tools Club, please check out our partnership page.

AI Tools Club

Read More