diff --git a/content/docs/best_practices/crawling.md b/content/docs/best_practices/crawling.md index a2ef476..40f0aa3 100644 --- a/content/docs/best_practices/crawling.md +++ b/content/docs/best_practices/crawling.md @@ -12,7 +12,7 @@ Colly's default configuration is optimized for scraping smaller number of sites ## Use persistent storage backend -By default Colly stores cookies and visited URLs in memory. You can replace the built-in in-memory storage backend with any custom backend. See more details [here](https://godoc.org/github.com/gocolly/colly/storage). +By default Colly stores cookies and visited URLs in memory. You can replace the built-in in-memory storage backend with any custom backend. See more details [here](https://godoc.org/github.com/gocolly/colly/v2/storage). ## Use async for long running jobs with recursive calls diff --git a/content/docs/best_practices/debugging.md b/content/docs/best_practices/debugging.md index eddcf85..f4abb35 100644 --- a/content/docs/best_practices/debugging.md +++ b/content/docs/best_practices/debugging.md @@ -11,13 +11,13 @@ Sometimes it's enough to place some `log.Println()` function calls to your callb ## Attach debugger to a collector -Attaching a basic logging debugger requires the `debug` (`github.com/gocolly/colly/debug`) package from Colly's repo. +Attaching a basic logging debugger requires the `debug` (`github.com/gocolly/colly/v2/debug`) package from Colly's repo. ```go import ( - "github.com/gocolly/colly" - "github.com/gocolly/colly/debug" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/debug" ) func main() { @@ -28,4 +28,4 @@ func main() { ## Implement a custom debugger -You can create any kind of custom debugger by implementing the [debug.Debugger](https://godoc.org/github.com/gocolly/colly/debug#Debugger) interface. A good example is [LogDebugger](https://github.com/gocolly/colly/blob/master/debug/logdebugger.go). +You can create any kind of custom debugger by implementing the [debug.Debugger](https://godoc.org/github.com/gocolly/colly/v2/debug#Debugger) interface. A good example is [LogDebugger](https://github.com/gocolly/colly/blob/master/debug/logdebugger.go). diff --git a/content/docs/best_practices/distributed.md b/content/docs/best_practices/distributed.md index e5e61d3..ed442d2 100644 --- a/content/docs/best_practices/distributed.md +++ b/content/docs/best_practices/distributed.md @@ -23,8 +23,8 @@ Colly has a built-in proxy switcher which rotates a list of proxies on every req package main import ( - "github.com/gocolly/colly" - "github.com/gocolly/colly/proxy" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/proxy" ) func main() { @@ -70,4 +70,4 @@ An example implementation can be found [here](/docs/examples/scraper_server). Visited URL and cookie data are stored in-memory by default. This is handy for short living scraper jobs, but it can be a serious limitation when dealing with large scale or long running crawling jobs. -Colly has the ability to replace the default in-memory storage with any storage backend which implements [colly/storage.Storage](https://godoc.org/github.com/gocolly/colly/storage#Storage) interface. Check out [existing storages](/docs/best_practices/storage). +Colly has the ability to replace the default in-memory storage with any storage backend which implements [colly/storage.Storage](https://godoc.org/github.com/gocolly/colly/v2/storage#Storage) interface. Check out [existing storages](/docs/best_practices/storage). diff --git a/content/docs/best_practices/extensions.md b/content/docs/best_practices/extensions.md index b685461..52a5f86 100644 --- a/content/docs/best_practices/extensions.md +++ b/content/docs/best_practices/extensions.md @@ -7,7 +7,7 @@ menu: weight: 50 --- -Extensions are small helper utilities shipped with Colly. List of plugins is available [here](https://godoc.org/github.com/gocolly/colly/extensions). +Extensions are small helper utilities shipped with Colly. List of plugins is available [here](https://godoc.org/github.com/gocolly/colly/v2/extensions). ## Usage @@ -18,8 +18,8 @@ The following example enables the random User-Agent switcher and the Referrer se import ( "log" - "github.com/gocolly/colly" - "github.com/gocolly/colly/extensions" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/extensions" ) func main() { diff --git a/content/docs/best_practices/storage.md b/content/docs/best_practices/storage.md index 795c59e..0e80205 100644 --- a/content/docs/best_practices/storage.md +++ b/content/docs/best_practices/storage.md @@ -7,13 +7,13 @@ menu: weight: 25 --- -Colly has an in-memory storage backend to store cookies and visited URLs, but it can be overwritten by any custom storage backend which implements [colly/storage.Storage](https://godoc.org/github.com/gocolly/colly/storage#Storage). +Colly has an in-memory storage backend to store cookies and visited URLs, but it can be overwritten by any custom storage backend which implements [colly/v2/storage.Storage](https://godoc.org/github.com/gocolly/colly/v2/storage#Storage). ## Existing Storage Backends -### [In-Memory Backend](https://godoc.org/github.com/gocolly/colly/storage#InMemoryStorage) +### [In-Memory Backend](https://godoc.org/github.com/gocolly/colly/v2/storage#InMemoryStorage) -The default backend of Colly. Use [collector.SetStorage()](https://godoc.org/github.com/gocolly/colly#Collector.SetStorage) to override. +The default backend of Colly. Use [collector.SetStorage()](https://godoc.org/github.com/gocolly/colly/v2#Collector.SetStorage) to override. ### [Redis backend](https://github.com/gocolly/redisstorage) diff --git a/content/docs/examples/basic.md b/content/docs/examples/basic.md index 92a234d..5d72907 100644 --- a/content/docs/examples/basic.md +++ b/content/docs/examples/basic.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/coursera_courses.md b/content/docs/examples/coursera_courses.md index 75911e8..704fdb2 100644 --- a/content/docs/examples/coursera_courses.md +++ b/content/docs/examples/coursera_courses.md @@ -15,7 +15,7 @@ import ( "os" "strings" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) // Course stores information about a coursera course diff --git a/content/docs/examples/cryptocoinmarketcap.md b/content/docs/examples/cryptocoinmarketcap.md index b58c3c0..f254735 100644 --- a/content/docs/examples/cryptocoinmarketcap.md +++ b/content/docs/examples/cryptocoinmarketcap.md @@ -14,7 +14,7 @@ import ( "log" "os" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/error_handling.md b/content/docs/examples/error_handling.md index 8c19fb4..60c0195 100644 --- a/content/docs/examples/error_handling.md +++ b/content/docs/examples/error_handling.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/factbase.md b/content/docs/examples/factbase.md index 41c3dd8..31bc8ac 100644 --- a/content/docs/examples/factbase.md +++ b/content/docs/examples/factbase.md @@ -15,7 +15,7 @@ import ( "io/ioutil" "strconv" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) var baseSearchURL = "https://factba.se/json/json-transcript.php?q=&f=&dt=&p=" diff --git a/content/docs/examples/google_groups.md b/content/docs/examples/google_groups.md index be81b3e..a4537fd 100644 --- a/content/docs/examples/google_groups.md +++ b/content/docs/examples/google_groups.md @@ -16,7 +16,7 @@ import ( "os" "strings" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) // Mail is the container of a single e-mail diff --git a/content/docs/examples/hackernews_comments.md b/content/docs/examples/hackernews_comments.md index cada626..dbdc1f4 100644 --- a/content/docs/examples/hackernews_comments.md +++ b/content/docs/examples/hackernews_comments.md @@ -17,7 +17,7 @@ import ( "strconv" "strings" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) type comment struct { diff --git a/content/docs/examples/instagram.md b/content/docs/examples/instagram.md index b47d85f..485f83c 100644 --- a/content/docs/examples/instagram.md +++ b/content/docs/examples/instagram.md @@ -19,7 +19,7 @@ import ( "os" "strings" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) // found in https://www.instagram.com/static/bundles/en_US_Commons.js/68e7390c5938.js diff --git a/content/docs/examples/login.md b/content/docs/examples/login.md index a3ab1e2..2d6d13d 100644 --- a/content/docs/examples/login.md +++ b/content/docs/examples/login.md @@ -12,7 +12,7 @@ package main import ( "log" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/max_depth.md b/content/docs/examples/max_depth.md index 0184558..e966656 100644 --- a/content/docs/examples/max_depth.md +++ b/content/docs/examples/max_depth.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/multipart.md b/content/docs/examples/multipart.md index 60d9c5d..35bdea9 100644 --- a/content/docs/examples/multipart.md +++ b/content/docs/examples/multipart.md @@ -16,7 +16,7 @@ import ( "os" "time" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func generateFormData() map[string][]byte { diff --git a/content/docs/examples/openedx_courses.md b/content/docs/examples/openedx_courses.md index 4f14b1e..5f6d644 100644 --- a/content/docs/examples/openedx_courses.md +++ b/content/docs/examples/openedx_courses.md @@ -15,7 +15,7 @@ import ( "strings" "time" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) // DATE_FORMAT default format date used in openedx diff --git a/content/docs/examples/parallel.md b/content/docs/examples/parallel.md index 682139e..b375141 100644 --- a/content/docs/examples/parallel.md +++ b/content/docs/examples/parallel.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/proxy_switcher.md b/content/docs/examples/proxy_switcher.md index 0398d5c..dea5593 100644 --- a/content/docs/examples/proxy_switcher.md +++ b/content/docs/examples/proxy_switcher.md @@ -13,8 +13,8 @@ import ( "bytes" "log" - "github.com/gocolly/colly" - "github.com/gocolly/colly/proxy" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/proxy" ) func main() { diff --git a/content/docs/examples/queue.md b/content/docs/examples/queue.md index fbb6246..d279de7 100644 --- a/content/docs/examples/queue.md +++ b/content/docs/examples/queue.md @@ -12,8 +12,8 @@ package main import ( "fmt" - "github.com/gocolly/colly" - "github.com/gocolly/colly/queue" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/queue" ) func main() { diff --git a/content/docs/examples/random_delay.md b/content/docs/examples/random_delay.md index be413a2..003027d 100644 --- a/content/docs/examples/random_delay.md +++ b/content/docs/examples/random_delay.md @@ -13,8 +13,8 @@ import ( "fmt" "time" - "github.com/gocolly/colly" - "github.com/gocolly/colly/debug" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/debug" ) func main() { diff --git a/content/docs/examples/rate_limit.md b/content/docs/examples/rate_limit.md index f39b703..8b77492 100644 --- a/content/docs/examples/rate_limit.md +++ b/content/docs/examples/rate_limit.md @@ -12,8 +12,8 @@ package main import ( "fmt" - "github.com/gocolly/colly" - "github.com/gocolly/colly/debug" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/debug" ) func main() { diff --git a/content/docs/examples/reddit.md b/content/docs/examples/reddit.md index 526d30a..a2acb89 100644 --- a/content/docs/examples/reddit.md +++ b/content/docs/examples/reddit.md @@ -14,7 +14,7 @@ import ( "os" "time" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) type item struct { diff --git a/content/docs/examples/redis_backend.md b/content/docs/examples/redis_backend.md index cd6ed07..5a9a38c 100644 --- a/content/docs/examples/redis_backend.md +++ b/content/docs/examples/redis_backend.md @@ -12,8 +12,8 @@ package main import ( "log" - "github.com/gocolly/colly" - "github.com/gocolly/colly/queue" + "github.com/gocolly/colly/v2" + "github.com/gocolly/colly/v2/queue" "github.com/gocolly/redisstorage" ) diff --git a/content/docs/examples/request_context.md b/content/docs/examples/request_context.md index 044dc0a..c7d5838 100644 --- a/content/docs/examples/request_context.md +++ b/content/docs/examples/request_context.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/scraper_server.md b/content/docs/examples/scraper_server.md index e1fb110..16d9518 100644 --- a/content/docs/examples/scraper_server.md +++ b/content/docs/examples/scraper_server.md @@ -14,7 +14,7 @@ import ( "log" "net/http" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) type pageInfo struct { diff --git a/content/docs/examples/shopify_sitemap.md b/content/docs/examples/shopify_sitemap.md index 6f1c8d4..5f7c8a2 100644 --- a/content/docs/examples/shopify_sitemap.md +++ b/content/docs/examples/shopify_sitemap.md @@ -12,7 +12,7 @@ package main import ( "fmt" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/url_filter.md b/content/docs/examples/url_filter.md index fb33fb1..9b38e09 100644 --- a/content/docs/examples/url_filter.md +++ b/content/docs/examples/url_filter.md @@ -13,7 +13,7 @@ import ( "fmt" "regexp" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/examples/xkcd_store.md b/content/docs/examples/xkcd_store.md index b619567..0525f66 100644 --- a/content/docs/examples/xkcd_store.md +++ b/content/docs/examples/xkcd_store.md @@ -14,7 +14,7 @@ import ( "log" "os" - "github.com/gocolly/colly" + "github.com/gocolly/colly/v2" ) func main() { diff --git a/content/docs/index.md b/content/docs/index.md index 49d9a0d..84ab4a2 100644 --- a/content/docs/index.md +++ b/content/docs/index.md @@ -12,4 +12,4 @@ Below you will find some of the most common and helpful pages from our documenta * [Getting started](/docs/introduction/start/) * [Basic scraper example](/docs/examples/basic/) -Also, check out [GoDoc](https://godoc.org/github.com/gocolly/colly) for API reference. +Also, check out [GoDoc](https://godoc.org/github.com/gocolly/colly/v2) for API reference. diff --git a/content/docs/introduction/configuration.md b/content/docs/introduction/configuration.md index 06fcc1b..3f5a2da 100644 --- a/content/docs/introduction/configuration.md +++ b/content/docs/introduction/configuration.md @@ -12,7 +12,7 @@ Colly is a highly customizable scraping framework. It has sane defaults and prov ## Collector configuration -Full list of collector attributes can be found [here](https://godoc.org/github.com/gocolly/colly#Collector). +Full list of collector attributes can be found [here](https://godoc.org/github.com/gocolly/colly/v2#Collector). The recommended way to initialize a collector is using `colly.NewCollector(options...)`. Create a collector with default settings: diff --git a/content/docs/introduction/start.md b/content/docs/introduction/start.md index e15c0cc..a89476b 100644 --- a/content/docs/introduction/start.md +++ b/content/docs/introduction/start.md @@ -16,7 +16,7 @@ Let's get started with some simple examples. First, you need to import Colly to your codebase: ```go -import "github.com/gocolly/colly" +import "github.com/gocolly/colly/v2" ``` @@ -32,7 +32,7 @@ c := colly.NewCollector() ## Callbacks -You can attach different type of callback functions to a `Collector` to control a collecting job or retrieve information. Check out the [related section](https://godoc.org/github.com/gocolly/colly#Collector.OnError) in the package documentation. +You can attach different type of callback functions to a `Collector` to control a collecting job or retrieve information. Check out the [related section](https://godoc.org/github.com/gocolly/colly/v2#Collector.OnError) in the package documentation. ### Add callbacks to a `Collector` diff --git a/themes/colly/layouts/_default/baseof.html b/themes/colly/layouts/_default/baseof.html index b3cd4ad..57f46ee 100755 --- a/themes/colly/layouts/_default/baseof.html +++ b/themes/colly/layouts/_default/baseof.html @@ -59,7 +59,7 @@ {{ range .Site.Menus.main.ByWeight }} {{ .Name }} {{ end }} - GoDoc + GoDoc
@@ -76,7 +76,7 @@ {{ .Name }} {{ end }}
- GoDoc + GoDoc GitHub