Blog / 实操指南
实操指南

如何使用 SERP API 抓取 Google 地图数据

在本分步指南中，学习如何使用 Bright Data 的 SERP API 抓取 Google 地图数据。
9 min read
favicon__brd
Kumar Harsh
谷歌地图抓取指南

作为 Google 最受欢迎的服务之一，Google 地图为企业与客户连接和扩展研究提供了巨大的机会。Google 地图拥有超过 2 亿个地点和企业的庞大数据库，为各种用途提供了丰富的数据源。

为了访问 Google 地图平台上的数据，Google 提供了 Google 地图 API。然而，该 API 可能会有一些限制且价格昂贵。在本文中，您将学习如何使用 Bright Data 的 SERP API 抓取 Google 地图数据，而无需依赖 Google 地图 API。

如何使用 Bright Data SERP API 抓取 Google 地图数据

开始使用 Bright Data SERP API 非常简单。在本节中，您将学习如何设置一个新的 Bright Data 帐户，注册 SERP API，并将其集成到您的 Node.js 应用程序中。

注册 SERP API

首先，前往 SERP API 主页，点击 免费试用 按钮：

SERP API 开始

接下来，在弹出的对话框中使用您的详细信息（或使用 Google 帐户）注册一个免费的 Bright Data 帐户：

注册

注册完成后，您将被重定向到您的 Bright Data 仪表板。在 代理和抓取基础设施 部分下点击 查看代理产品 按钮：

Bright Data 仪表板

在 代理和抓取基础设施 页面上，点击 开始使用 按钮：

SERP API仪表板

在下一页，您将看到 SERP API 的详细信息，以及估计的每月费用。您的试用帐户应该有足够的积分来支付您的初始请求。

在此页面上，为您的 API 实例选择一个名称（例如“maps”）。对于 广告模式 问题，保持为 关闭。广告模式选项在您想要包含或排除 Google 搜索结果页面的广告时很有用。

此外，暂时关闭 异步请求 以便于本教程的演示。在将 API 实现到您的应用程序中时，如果您的系统支持异步处理，可以考虑打开它。特别是在处理大数据对象时，它可以提高系统的效率。

完成后，点击 添加 按钮：

SERP API控制面板

此时，您已经创建了一个新的 SERP API 实例。现在，您可以看到用于访问 API 的凭证。门户还提供了一个示例 curl 请求，您可以将其复制并在终端中使用，以立即测试新的凭证：

凭证

在 Bright Data 平台上的设置到此完成。请务必记下屏幕上提供的代理主机、用户名和密码，因为您需要它们来从您的 Node.js/Python 应用程序访问 SERP API。

在 Node.js 中使用 SERP API

要在 Node.js 中使用 SERP API，请先创建一个新目录并在其中初始化一个新的 Node.js 项目：

mkdir serp-node

cd serp-node

npm init

选择默认值创建项目。项目创建后，创建一个新文件，index.js，以存储您的 Node.js 脚本：

touch index.js

在保存脚本之前，运行以下命令安装 request-promise 模块，以便轻松进行 HTTP 请求：

npm install request-promise

将以下代码存储在文件中（确保使用您的 SERP API 实例值更新代理详细信息）：

const request = require('request-promise');


// Define parameters to pass into the API request
const keyword = process.argv[2]
const username = 'xxxx';    // your proxy username
const password = 'xxxx';    // your proxy password
const port = 0000;          // your proxy port
const host = 'xxxx'         // your proxy host
const session_id = (1000000 * Math.random())|0;
const super_proxy = 'http://'+username+'-session-'+session_id+':'+password+'@'+ host + ':'+port;
const options = {
    url: 'https://www.google.com/maps/search/' + keyword.replace(' ', '+') + '/?gl=us&lum_json=1',
    proxy: super_proxy,
    rejectUnauthorized: false
};

// Send request to the Maps search API
request(options)
.then(function(data){

    // Parse the response as JSON
    const result = JSON.parse(data)

    // Print the response JSON
    console.log("Printing Maps search results...")
    console.log(result)

    // Extract the  f_id of the first search result
    const f_id = result.organic[0].fid

    // Define API request parameters for retrieving reviews for the search result
    var options = {
        url: 'https://www.google.com/reviews?fid=' + f_id + '&lum_json=1',
        proxy: super_proxy,
        rejectUnauthorized: false
    };


    // Send request to the Reviews API
    request(options)
    .then(function(data){
        console.log("Printing Google Reviews results...")

        // Parse and print the response
        console.log(JSON.stringify(JSON.parse(data), null, 2))
    },
    function(err){ console.error(err); })

},
    function(err){ console.error(err); });

该脚本将搜索关键字作为命令行参数。保存文件后，使用以下命令运行它：

node index.js "hotels california"

输出如下所示：

Printing Maps search results...
{
  general: {
    query: 'hotels california',
    language: 'en',
    country: 'United States',
    country_code: 'US',
    gl: 'US'
  },
  organic: [
    {
      "title": "Freehand Los Angeles",
      "display_link": "freehandhotels.com",
      "link": "https://freehandhotels.com/los-angeles/?utm_source=google-my-business&utm_medium=organic&utm_campaign=hotel-los-angeles",
      "address": "416 W 8th St, Los Angeles, CA 90014",
      "phone": "+12136120021",
      "category": [
        {
          "title": "4-star hotel"
        },
        {
          "id": "hotel",
          "title": "Hotel"
        }
      ],
      "tags": [
        {
          "group_id": "hotel_amenities",
          "group_title": "Hotel amenities",
          "key_id": 21,
          "value_title": "Pool"
        },
        {
          "group_id": "hotel_amenities",
          "group_title": "Hotel amenities",
          "key_id": 22,
          "value_title": "Free Wi-Fi"
        },
        {
          "group_id": "hotel_amenities",
          "group_title": "Hotel amenities",
          "key_id": 2,
          "value_title": "Air-conditioned"
        },
        {
          "group_id": "hotel_amenities",
          "group_title": "Hotel amenities",
          "key_id": 10,
          "value_title": "Breakfast"
        }
      ],
      "summary": "Hip hotel with a rooftop pool & bar",
      "description": "Trendy dorms & rooms in a funky lodging with dining & breakfast, plus a rooftop pool & bar.",
      "rating": 4.1,
      "reviews_cnt": 2263,
      "latitude": 34.044755099999996,
      "longitude": -118.2564556,
      "claimed": true,
      "fid": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
      "map_id_encoded": "ChIJE80IjrXHwoARZo6wiQusTrc",
      "map_id": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
      "map_link": "https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
      "original_image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w408-h272-k-no",
      "image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
      "thumbnail": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w128-h86-k-no",
      "icon": "https://lh5.googleusercontent.com/-1FySCxGYcRE/AAAAAAAAAAI/AAAAAAAAAAA/cD1i5xM2CPs/s44-p-k-no-ns-nd/photo.jpg",
      "image_url": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
      "rank": 1
    }
    // 19 records omitted
  ],
  ads: [
    {
      title: 'Hampton Inn & Suites Rohnert Park - Sonoma County',
      address: '6248 Redwood Dr, Rohnert Park, CA 94928',
      category: [Array],
      tags: [Array],
      summary: 'Low-key hotel with a pool & a gym',
      description: 'Unpretentious hotel offering an outdoor pool & a fitness room, plus parking & breakfast.',
      rating: 4.4,
      reviews_cnt: 215,
      latitude: 38.350170999999996,
      longitude: -122.71469599999999,
      claimed: true,
      fid: '0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
      map_id_encoded: 'ChIJ7WG1cglKhIARzOfAOsHEFR0',
      map_id: '0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
      map_link: 'https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80844a0972b561ed:0x1d15c4c13ac0e7cc',
      original_image: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w408-h272-k-no',
      image: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w138-h92-k-no',
      thumbnail: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w129-h86-k-no',
      image_url: 'https://lh3.googleusercontent.com/gps-proxy/AE4_-5FpvLkXD1XpPW4nu3lyfKyKBIRFpSq4eqdABiJhNpxR3mm1nXC2a4WD-CxpFwGIe3FZqXr1gnRJd-QpgqE-sRCE8zu7LeRXzUdGWCSr3yTJURar4MS0duJxol4JQ9e3LR6jFWxtzwrvAuXmasAdUhefZSQzU3_OM1K46SyCXrX819IcmStXtSqzYw=w138-h92-k-no',
      rank: 1
    },
    // 1 record omitted
  ]
}
Printing Google Reviews results...
{
  "reviews": [
    {
      "reviewer": {
        "profile_photo_url": "https://lh3.googleusercontent.com/a-/AD_cMMTCnapnhVKwImX7-W1lGGDw8-jYDbjZ7bi7EQcRjQ=s40-c-c0x00000000-cc-rp-mo-ba3-br100",
        "display_name": "Leroy T"
      },
      "rating": "4/5",
      "review_reply": "Thank you for your review, Leroy! It is great to see that you enjoyed your time with us. …",
      "comment": "Overall it was a great stay, i had the deluxe king corner one. Which I suggest go to their website to book where you can select a corner unit.I rated 4 out of 5 because i feel that they can improve by having breakfast included, kettle/coffee maker, complimentary water, toothbrush toothpaste but I guess since I'm writing this review at least future clients will be aware.As far as safety, the elevator and 24/7 door by the elevator is card key access. So felt pretty safe.Costumer service was great smooth checkin and out even the housekeeper was nice.Facilities are nice the pool hours are in the pictures didn't use the gym so I can't speak about that.",
      "photos": [
        "https://lh5.googleusercontent.com/p/AF1QipPfJceU9nK8p0FhUDl0mIObGUC1AoEF2bZMDDs6=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipPCpi4ED4kk7FFvDxdh9HrsB60Pr01sUkL3FP_o=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipMjNO_OAG8QA74JszAAE2K6gicxbGEMAQbRXtwt=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipMjDjq7wP8iUbUiFeBNFHjzOFBr6aVUPDcfDSHe=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNcccwmp8K24QID0jU9PcyWq2QswyyK6oc6gf1d=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNCnCfWeAdQ7JUbZHzFL_myn_S9Uv6rIiq7e6bV=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNtFZMp7PZNr7e9OabzzVeGWzadGsgW7DUY2pcC=w100-h100-p-n-k-no"
      ],
      "created": "5 months ago on Google",
      "star_rating": "4/5"
    },
    // More records omitted
  ]

理解您检索到的信息

本示例中使用的脚本遵循以下结构：

  1. 检索给定关键字的 Google 地图搜索结果
  2. 访问并检索给定关键字的第一个搜索结果的评论

以下是从 SERP API 检索到的搜索结果示例：

{
  "title": "Freehand Los Angeles",
  "display_link": "freehandhotels.com",
  "link": "https://freehandhotels.com/los-angeles/?utm_source=google-my-business&utm_medium=organic&utm_campaign=hotel-los-angeles",
  "address": "416 W 8th St, Los Angeles, CA 90014",
  "phone": "+12136120021",
  "category": [
    {
      "title": "4-star hotel"
    },
    {
      "id": "hotel",
      "title": "Hotel"
    }
  ],
  "tags": [
    {
      "group_id": "hotel_amenities",
      "group_title": "Hotel amenities",
      "key_id": 21,
      "value_title": "Pool"
    },
    {
      "group_id": "hotel_amenities",
      "group_title": "Hotel amenities",
      "key_id": 22,
      "value_title": "Free Wi-Fi"
    },
    {
      "group_id": "hotel_amenities",
      "group_title": "Hotel amenities",
      "key_id": 2,
      "value_title": "Air-conditioned"
    },
    {
      "group_id": "hotel_amenities",
      "group_title": "Hotel amenities",
      "key_id": 10,
      "value_title": "Breakfast"
    }
  ],
  "summary": "Hip hotel with a rooftop pool & bar",
  "description": "Trendy dorms & rooms in a funky lodging with dining & breakfast, plus a rooftop pool & bar.",
  "rating": 4.1,
  "reviews_cnt": 2263,
  "latitude": 34.044755099999996,
  "longitude": -118.2564556,
  "claimed": true,
  "fid": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
  "map_id_encoded": "ChIJE80IjrXHwoARZo6wiQusTrc",
  "map_id": "0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
  "map_link": "https://www.google.com/maps/place/data=!3m1!4b1!4m2!3m1!1s0x80c2c7b58e08cd13:0xb74eac0b89b08e66",
  "original_image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w408-h272-k-no",
  "image": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
  "thumbnail": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w128-h86-k-no",
  "icon": "https://lh5.googleusercontent.com/-1FySCxGYcRE/AAAAAAAAAAI/AAAAAAAAAAA/cD1i5xM2CPs/s44-p-k-no-ns-nd/photo.jpg",
  "image_url": "https://lh5.googleusercontent.com/p/AF1QipOmiTy09SlFk0wu-kXofG9Nfu_sz0Pg6ffFjBry=w137-h92-k-no",
  "rank": 1
}

正如您所见，API 返回了搜索结果（即一个地点）的所有基本信息，包括其标题、链接、地址和联系方式。此外，API 还返回了其他详细信息，如设施、评论摘要、评分、坐标、图片和类别详情。

以下是使用 SERP 检索到的评论示例：

{
      "reviewer": {
        "profile_photo_url": "https://lh3.googleusercontent.com/a-/AD_cMMTCnapnhVKwImX7-W1lGGDw8-jYDbjZ7bi7EQcRjQ=s40-c-c0x00000000-cc-rp-mo-ba3-br100",
        "display_name": "Leroy T"
      },
      "rating": "4/5",
      "review_reply": "Thank you for your review, Leroy! It is great to see that you enjoyed your time with us. …",
      "comment": "Overall it was a great stay, i had the deluxe king corner one. Which I suggest go to their website to book where you can select a corner unit.I rated 4 out of 5 because i feel that they can improve by having breakfast included, kettle/coffee maker, complimentary water, toothbrush toothpaste but I guess since I'm writing this review at least future clients will be aware.As far as safety, the elevator and 24/7 door by the elevator is card key access. So felt pretty safe.Costumer service was great smooth checkin and out even the housekeeper was nice.Facilities are nice the pool hours are in the pictures didn't use the gym so I can't speak about that.",
      "photos": [
        "https://lh5.googleusercontent.com/p/AF1QipPfJceU9nK8p0FhUDl0mIObGUC1AoEF2bZMDDs6=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipPCpi4ED4kk7FFvDxdh9HrsB60Pr01sUkL3FP_o=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipMjNO_OAG8QA74JszAAE2K6gicxbGEMAQbRXtwt=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipMjDjq7wP8iUbUiFeBNFHjzOFBr6aVUPDcfDSHe=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNcccwmp8K24QID0jU9PcyWq2QswyyK6oc6gf1d=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNCnCfWeAdQ7JUbZHzFL_myn_S9Uv6rIiq7e6bV=w100-h100-p-n-k-no",
        "https://lh5.googleusercontent.com/p/AF1QipNtFZMp7PZNr7e9OabzzVeGWzadGsgW7DUY2pcC=w100-h100-p-n-k-no"
      ],
      "created": "5 months ago on Google",
      "star_rating": "4/5"
    },

每条由 SERP API 检索到的评论都包含完整的评论详细信息，包括评论内容、回复评论、图片（如果有）、评分和基本的评论者详细信息，如姓名和照片。这些数据对象足以帮助您了解一家企业根据用户在 Google 地图上留下的评论的表现。

结论

虽然 Google 地图 API 是将 Google 地图嵌入到您应用中的一个很好的工具，但它不是访问 Google 地图数据的最佳选择。由于存在速率限制、价格高昂和数据点有限等问题，选择一个可以提供更好体验且成本较低的专用 API 是合理的。

在本文中，您看到了 Bright Data 的 SERP API 如何优于 Google 地图 API 提供的 Places API 用于检索 Google 地图数据。您还看到了如何在自己的项目中设置它。

本文应能帮助您轻松有效地从 Google 地图中提取所需的信息。请确保今天注册您的 SERP API 免费试用。

免费试用
Table of Contents

    你也可能对此有兴趣

    如何使将网页抓取用于机器学习
    网络数据
    Antonello-Zanini
    Antonello Zanini

    如何使将网页抓取用于机器学习

    学习如何为机器学习项目收集、准备和使用网络抓取的数据，包括 ETL 设置和模型训练技巧。
    10-12 月-2024
    10 min read
    如何使用 Python 抓取 Google Maps
    网络数据
    Antonello-Zanini
    Antonello Zanini

    如何使用 Python 抓取 Google Maps

    学习如何使用 Python 抓取 Google Maps，从设置到结果导出，详细操作一网打尽，并了解如何借助 Bright Data 的 API 应对棘手挑战。
    10-12 月-2024
    9 min read
    如何在 Python 中抓取 GitHub 存储库
    实操指南
    Antonello-Zanini
    Antonello Zanini

    如何在 Python 中抓取 GitHub 存储库

    是否考虑过如何抓取 GitHub？从这里找到答案！
    09-12 月-2024
    6 min read