2026-05-21 16:01
🌟 Exciting updates in user-sequence data management! The recent article highlights a redesign of the user-sequence platform aimed at enhancing cost-efficiency, speed, and ease of use. User sequences, which track recent actions and enrichments, are crucial for ML models in ranking, retrieval, and recommendation systems. Key improvements include a shared execution engine, configuration-as-code, and a lambda architecture to balance freshness and completeness. This redesign supports multiple...
Source: Pinterest Engineering
Pinterest Engineering
2026-05-12 16:01
🚀 The tech industry is evolving with AI agents, but consistent skill invocation can be challenging. In a recent article, Daniel Reed discusses testing processes to enhance agent performance. By experimenting with Pin-agent and Claude Code, insights were gained into improving skill invocation rates. A reliable test harness, including a Bash script, was developed to run positive and negative prompts. Results showed that clearer, more detailed prompts significantly improved accuracy rates. For...
Source: Pinterest Engineering
Pinterest Engineering
2026-05-08 19:01
Enhancing ad relevance is key for effective advertising. A recent article discusses the integration of real-time context into sequential recommender models. The new Contextual Sequential Two Tower Model combines historical user data with current browsing context, improving ad recommendations on platforms like Related Pins. This method has significantly boosted candidate relevance and user engagement. 📈🛍️ Key results show a 3x to 10x increase in Recall@K and a measurable lift in Return on Ad...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-27 16:01
🔍 At Pinterest, optimizing conversion ads is essential for connecting users with products they want to buy. The challenge lies in offsite conversions, which are less frequent and harder to track compared to onsite engagement. 📈 In 2023, Pinterest launched a dedicated candidate generation model designed specifically for conversions. This model has led to increased clickthrough rates and improved advertiser performance, with further enhancements noted in 2025. 🔑 Key strategies included a multi-...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-20 16:01
📌 Pinterest has developed the Minimal Important Query Param Set (MIQPS) algorithm to address URL normalization challenges. This algorithm helps deduplicate content by identifying which URL parameters are essential for content identity, improving efficiency in processing millions of URLs. It adapts dynamically to various merchant domains, ensuring consistent catalog organization and user experience. Learn more about how MIQPS enhances content quality at scale! 🌐🔗 #Pinterest #URLNormalization...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-15 16:01
In early 2025, Pinterest's Kubernetes team faced crashing training jobs on their ML platform due to network connectivity issues. A three-month investigation revealed CPU bottlenecks linked to the AWS network driver and excessive memory cgroups, dubbed "zombies." This impacted system performance, leading to job failures. The issue was traced back to a crashing ECS agent on GPU instances, which created numerous memory cgroups. Disabling this agent stabilized the system. #Tech #Engineering...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-13 19:01
Scaling recommendation systems at Pinterest involves significant advancements in quality and efficiency. The team has achieved a 100x increase in model parameters, but this creates infrastructure challenges. To manage costs, they implemented request-level deduplication, which optimizes data processing and storage by eliminating redundancy. This technique enhances storage efficiency, speeds up training, and improves serving throughput. Key outcomes include 10-50x storage compression, 4x...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-08 16:01
📱 Performance is key in mobile apps, and Pinterest is dedicated to improving it across all user experiences like the "Home Feed" and "Search Result Feed." 🔍 User perceived latency, or "Visually Complete," measures the time from user action to content display. It varies by app and surface, requiring tailored measurement logic, which can be resource-intensive for engineers. 🌟 To streamline this, Pinterest has integrated Visually Complete logic into a base UI class, allowing automatic tracking...
Source: Pinterest Engineering
Pinterest Engineering
2026-04-07 16:01
📢 Exciting updates from Pinterest's Home Feed! The evolution of their multi-objective optimization focuses on enhancing user engagement by improving feed recommendation systems. Key strategies include balancing short-term actions with long-term user satisfaction through advanced algorithms like Determinantal Point Process (DPP) and Sliding Spectrum Decomposition (SSD). These enhancements aim to diversify content, ensuring a more satisfying user experience. Pinterest's ongoing efforts will...
Source: Pinterest Engineering
Pinterest Engineering
2026-03-30 16:01
At Pinterest, we are upgrading our ML stack from PyTorch 2.1 to 2.6 to harness improvements like better GPU support and enhanced training efficiency. 🚀 This upgrade involves navigating challenges such as outdated Ubuntu versions, breaking API changes, and ensuring zero downtime during the transition. We are carefully managing dependencies and testing at each stage to maintain performance. 🔧 Our journey highlights the importance of thorough planning and collaboration. #MachineLearning #PyTorch...
Source: Pinterest Engineering
Pinterest Engineering
2026-03-19 16:01
Over the past year, Pinterest has developed a robust Model Context Protocol (MCP) ecosystem. This open-source standard allows AI agents to automate engineering tasks efficiently. MCP servers are hosted internally, optimizing security and performance. A centralized registry helps teams manage and discover approved servers. Notable servers include Presto for data access and Spark for debugging. The system emphasizes security, ensuring only authorized users can access sensitive tools. In January...
Source: Pinterest Engineering
Pinterest Engineering
2026-03-06 22:01
🚀 Pinterest has developed a powerful Analytics Agent to enhance Text-to-SQL capabilities. This system transforms analyst queries into meaningful representations, allowing for better understanding of analytical intent. It also uses structured patterns and governance-aware ranking to ensure trustworthy results. With over 100,000 analytical tables, this solution streamlines data exploration, enabling faster and more accurate SQL generation for analysts. #DataAnalytics #TextToSQL #AI #Pinterest...
Source: Pinterest Engineering
Pinterest Engineering
2026-03-03 20:01
📊 Pinterest has developed a unified ads engagement model to enhance ad predictions across various surfaces like Home Feed and Search. Previously, separate models created inefficiencies in iteration and costs. The new approach consolidates these systems while allowing for surface-specific features. Key strategies involved starting simple, iterating gradually, and ensuring safe deployment. Initial tests showed promising improvements in performance and efficiency. Learn more about this...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-27 17:01
📊 Exploring the Online–Offline (O/O) discrepancy in Pinterest's L1 conversion models reveals crucial insights. While offline metrics showed significant gains in loss and calibration, online A/B testing displayed neutral or negative results. The investigation focused on features, embeddings, and funnel design, identifying key issues like feature coverage and version skew. This analysis emphasizes the importance of aligning offline and online metrics for successful model deployment. #Pinterest...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-24 17:01
Introducing Piqama, Pinterest's new Quota Management Ecosystem! 🌟 Piqama efficiently manages various resources like memory, CPU, and QPS. It offers a user-friendly portal for seamless quota lifecycle management, including updates and predictions. This system supports both capacity management for Big Data and rate-limiting for online services. Key features include quota schema management, validation frameworks, and customizable enforcement strategies. Piqama aims to optimize resource...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-17 17:01
Pinterest has introduced a feature called Auto Memory Retries to significantly reduce out-of-memory (OOM) errors in their Apache Spark applications. 🚀 This feature automatically identifies tasks that require more memory and retries them on larger executors, improving resource management. Pinterest processes over 90,000 Spark jobs daily, making this enhancement crucial for performance and efficiency. 🔍 The result? A remarkable 96% drop in OOM failures, freeing up resources and reducing delays...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-13 23:44
🚀 Pinterest has launched a GPU-serving two-tower model for ad engagement prediction. This model enhances lightweight ranking in the ad recommendation system by narrowing down candidates efficiently. The new architecture combines Multi-gate Mixture-of-Experts (MMOE) with Deep & Cross Networks (DCN). This shift has resulted in a 5–10% reduction in offline loss for click-through rate predictions. Improvements in GPU training efficiency have also doubled model iteration speed. #Pinterest...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-05 17:01
🚀 Pinterest is redefining its database ingestion framework to meet growing demands for real-time data. The new system addresses significant challenges from legacy batch workflows, improving latency, efficiency, and compliance. Key enhancements include a unified CDC-based framework utilizing Kafka, Flink, and Spark, ensuring timely access to data changes. Stay tuned for more insights on automated schema evolution in the next installment! 📊🔍 #DataIngestion #PinterestEngineering #RealTimeData...
Source: Pinterest Engineering
Pinterest Engineering
2026-02-02 17:01
🚀 The article "Beyond Two Towers" discusses the evolution of ad ranking models from a traditional Two-Tower architecture to a more advanced GPU-based system. 📈 The shift aims to enhance recommendation quality by allowing deeper user-item interactions, overcoming limitations of the old model. 💡 Key optimizations include bundling features with models, moving business logic into the model, and rethinking data flow to minimize latency. Stay tuned for more insights on this innovative approach!...
Source: Pinterest Engineering
Pinterest Engineering
2026-01-28 23:01
Pinterest is enhancing ad relevance through Behavioral Sequence Modeling. This approach uses historical user behavior to predict future interactions with advertisers. The model focuses on personalizing ad candidates by analyzing user actions like views and purchases. Key metrics such as Recall@K help evaluate performance, showing significant improvements in conversion and cost efficiency since launch. Exciting advancements are also underway to predict specific product interactions for a more...
Source: Pinterest Engineering
Pinterest Engineering
2026-01-13 20:03
Introducing PinLanding, a new AI-driven pipeline designed to create shopping collections from billions of products. 🛍️ This system leverages multimodal language models to understand user search patterns and generate collections based on product content. Key components include analyzing shopping intents, building a vocabulary of attributes, and constructing feeds for enhanced search precision. Initial results show a significant increase in unique shopping topics and improved search...
Source: Pinterest Engineering
Pinterest Engineering
2025-12-08 17:02
Pinterest has developed an AI-driven system to monitor violative content in real-time. This approach, known as prevalence measurement, assesses the percentage of views on policy-violating content daily. Historically, user reports were the main metric, but this method left gaps. Under-reported issues, such as self-harm, and rare content types can go unnoticed. The new system samples user impressions, allowing for a broader, more stable view of content violations. This provides quicker insights...
Source: Pinterest Engineering
Pinterest Engineering
2025-12-05 20:02
Pinterest is enhancing content quality by utilizing user feedback through surveys. 📊 In partnership with the Inspired Internet Pledge, the platform collects ratings on visual appeal to understand what users value. This data is then used to train machine learning models to improve recommendations across Homefeed, Related Pins, and Search. The initiative aims to reduce low-quality content and elevate user experience. 📈✨ Learn more about how Pinterest is putting Pinners first! #Pinterest...
Source: Pinterest Engineering
Pinterest Engineering
2025-12-04 17:02
The AI landscape is shifting, highlighting the importance of open-source models. Pinterest’s team reports significant cost savings, achieving similar performance to proprietary models for under 10% of the cost. The focus is on tailored models that excel in specific tasks, leveraging Pinterest's unique data. This trend reflects a broader industry move towards domain-specific data and personalization. As Pinterest continues to invest in open-source AI, the goal remains to enhance user...
Source: Pinterest Engineering
Pinterest Engineering
2025-12-03 17:02
At Pinterest, we are enhancing our observability tools to create a unified experience. Traditionally, our systems operated in silos, making it hard to connect logs, metrics, and traces. To address this, we are implementing the Model Context Protocol (MCP) to streamline our observability data. This will enable faster root-cause analysis and empower our teams to build context-aware tools. We are excited to embrace AI in this journey, aiming for a more intelligent observability future! 🚀🔍...
Source: Pinterest Engineering
Pinterest Engineering
2025-11-10 23:02
🚀 Exciting updates from Pinterest Engineering! To address slow and flaky Android testing builds, the team implemented a runtime-aware sharding mechanism. This new approach reduces build times by 36%, cutting down the slowest shard's runtime by 55%. The solution optimizes test distribution based on historical data, ensuring balanced runtimes across shards. 📉 This advancement enhances developer velocity and streamlines the CI process. #PinterestEngineering #CIPipeline #AndroidDevelopment...
Source: Pinterest Engineering
Pinterest Engineering
2025-11-04 18:01
Over the past decade, Pinterest has transformed its AI Platform from fragmented machine learning stacks to a unified system that supports all major operations. Key lessons include the importance of organizational alignment for adoption, the need for layered foundations, and the interplay of enablement, efficiency, and velocity. The evolution is divided into five eras, highlighting the journey from initial fragmentation to broader alignment and advanced capabilities. For more insights, check...
Source: Pinterest Engineering
Pinterest Engineering
2025-10-21 21:42
📌 Pinterest is enhancing its platform by introducing user journeys, focusing on understanding users' long-term goals beyond immediate interests. A user journey combines interests, intent, and context, enabling personalized recommendations for projects like wedding planning or home renovations. To implement this, Pinterest is using a dynamic keyword extraction approach for greater adaptability and personalization. This shift aims to help users achieve their aspirations effectively. #Pinterest...
Source: Pinterest Engineering
Pinterest Engineering
2025-10-17 16:01
🔍 Pinterest recently tackled a significant challenge during a PyTorch upgrade, experiencing a 50% drop in ML training throughput. The team meticulously traced the issue, identifying low-level Linux kernels and a monitoring process as major culprits. Their systematic debugging provided insights into optimizing performance and enhancing training efficiency. This journey highlights the importance of thorough analysis and innovative solutions in tackling complex tech issues. 💻✨ #MachineLearning...
Source: Pinterest Engineering
Pinterest Engineering
2025-09-10 16:01
Pinterest is evolving its data processing capabilities with Moka, a next-gen platform built on AWS EKS. 🌐 The new infrastructure includes standardized cluster environments like test, dev, staging, and production, allowing for effective resource management and security. Key features include enhanced logging using Fluent Bit and observability metrics via OTEL, improving insights into performance and stability. 📊 Learn more about Moka's architecture and its future developments. #DataProcessing...
Source: Pinterest Engineering
Pinterest Engineering
2025-08-22 20:12
🚀 Pinterest has introduced PinConsole, an Internal Developer Platform (IDP) aimed at simplifying the developer experience. This initiative addresses increasing complexity and improves engineering velocity for over 550 million users. 🔍 The team identified challenges such as tool fragmentation and inconsistent workflows, which were hindering productivity. By leveraging Backstage, PinConsole creates a unified interface, allowing engineers to focus on business logic. 📈 Early adoption shows...
Source: Pinterest Engineering
Pinterest Engineering