rebelonion/Dantotsu
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial commit

Browse source
This commit is contained in:
Finnley Somdahl 2023-10-17 18:42:43 -05:00
commit 21bfbfb139
520 changed files with 47819 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

176
app/src/main/java/eu/kanade/tachiyomi/LICENSE Normal file
View file

@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS

3
app/src/main/java/eu/kanade/tachiyomi/NOTICE.md Normal file
View file

@ -0,0 +1,3 @@
NOTICE
This software includes code modified from Aniyomi, available at https://github.com/aniyomiorg/aniyomi/.

9
app/src/main/java/eu/kanade/tachiyomi/animesource/ConfigurableAnimeSource.kt Normal file
View file

@ -0,0 +1,9 @@
package eu.kanade.tachiyomi.animesource
import ani.dantotsu.aniyomi.animesource.AnimeSource
import ani.dantotsu.aniyomi.PreferenceScreen
interface ConfigurableAnimeSource : AnimeSource {
fun setupPreferenceScreen(screen: PreferenceScreen)
}

410
app/src/main/java/eu/kanade/tachiyomi/animesource/online/AnimeHttpSource.kt Normal file
View file

@ -0,0 +1,410 @@
package eu.kanade.tachiyomi.animesource.online
import ani.dantotsu.aniyomi.animesource.AnimeCatalogueSource
import eu.kanade.tachiyomi.animesource.model.AnimeFilterList
import eu.kanade.tachiyomi.animesource.model.AnimesPage
import eu.kanade.tachiyomi.animesource.model.SAnime
import eu.kanade.tachiyomi.animesource.model.SEpisode
import eu.kanade.tachiyomi.animesource.model.Video
import eu.kanade.tachiyomi.network.GET
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.asObservableSuccess
import eu.kanade.tachiyomi.network.newCachelessCallWithProgress
import okhttp3.Headers
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response
import rx.Observable
import uy.kohesive.injekt.injectLazy
import java.net.URI
import java.net.URISyntaxException
import java.security.MessageDigest
import java.util.concurrent.TimeUnit
/**
* A simple implementation for sources from a website.
*/
abstract class AnimeHttpSource : AnimeCatalogueSource {
/**
* Network service.
*/
protected val network: NetworkHelper by injectLazy()
/**
* Base url of the website without the trailing slash, like: http://mysite.com
*/
abstract val baseUrl: String
/**
* Version id used to generate the source id. If the site completely changes and urls are
* incompatible, you may increase this value and it'll be considered as a new source.
*/
open val versionId = 1
/**
* Id of the source. By default it uses a generated id using the first 16 characters (64 bits)
* of the MD5 of the string: sourcename/language/versionId
* Note the generated id sets the sign bit to 0.
*/
override val id by lazy {
val key = "${name.lowercase()}/$lang/$versionId"
val bytes = MessageDigest.getInstance("MD5").digest(key.toByteArray())
(0..7).map { bytes[it].toLong() and 0xff shl 8 * (7 - it) }.reduce(Long::or) and Long.MAX_VALUE
}
/**
* Headers used for requests.
*/
val headers: Headers by lazy { headersBuilder().build() }
/**
* Default network client for doing requests.
*/
open val client: OkHttpClient
get() = network.client
/**
* Headers builder for requests. Implementations can override this method for custom headers.
*/
protected open fun headersBuilder() = Headers.Builder().apply {
add("User-Agent", network.defaultUserAgentProvider())
}
/**
* Visible name of the source.
*/
override fun toString() = "$name (${lang.uppercase()})"
/**
* Returns an observable containing a page with a list of anime. Normally it's not needed to
* override this method.
*
* @param page the page number to retrieve.
*/
override fun fetchPopularAnime(page: Int): Observable<AnimesPage> {
return client.newCall(popularAnimeRequest(page))
.asObservableSuccess()
.map { response ->
popularAnimeParse(response)
}
}
/**
* Returns the request for the popular anime given the page.
*
* @param page the page number to retrieve.
*/
protected abstract fun popularAnimeRequest(page: Int): Request
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
protected abstract fun popularAnimeParse(response: Response): AnimesPage
/**
* Returns an observable containing a page with a list of anime. Normally it's not needed to
* override this method.
*
* @param page the page number to retrieve.
* @param query the search query.
* @param filters the list of filters to apply.
*/
override fun fetchSearchAnime(page: Int, query: String, filters: AnimeFilterList): Observable<AnimesPage> {
return Observable.defer {
try {
client.newCall(searchAnimeRequest(page, query, filters)).asObservableSuccess()
} catch (e: NoClassDefFoundError) {
// RxJava doesn't handle Errors, which tends to happen during global searches
// if an old extension using non-existent classes is still around
throw RuntimeException(e)
}
}
.map { response ->
searchAnimeParse(response)
}
}
/**
* Returns the request for the search anime given the page.
*
* @param page the page number to retrieve.
* @param query the search query.
* @param filters the list of filters to apply.
*/
protected abstract fun searchAnimeRequest(page: Int, query: String, filters: AnimeFilterList): Request
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
protected abstract fun searchAnimeParse(response: Response): AnimesPage
/**
* Returns an observable containing a page with a list of latest anime updates.
*
* @param page the page number to retrieve.
*/
override fun fetchLatestUpdates(page: Int): Observable<AnimesPage> {
return client.newCall(latestUpdatesRequest(page))
.asObservableSuccess()
.map { response ->
latestUpdatesParse(response)
}
}
/**
* Returns the request for latest anime given the page.
*
* @param page the page number to retrieve.
*/
protected abstract fun latestUpdatesRequest(page: Int): Request
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
protected abstract fun latestUpdatesParse(response: Response): AnimesPage
/**
* Returns an observable with the updated details for a nanime. Normally it's not needed to
* override this method.
*
* @param anime the anime to be updated.
*/
override fun fetchAnimeDetails(anime: SAnime): Observable<SAnime> {
return client.newCall(animeDetailsRequest(anime))
.asObservableSuccess()
.map { response ->
animeDetailsParse(response).apply { initialized = true }
}
}
/**
* Returns the request for the details of an anime. Override only if it's needed to change the
* url, send different headers or request method like POST.
*
* @param anime the anime to be updated.
*/
open fun animeDetailsRequest(anime: SAnime): Request {
return GET(baseUrl + anime.url, headers)
}
/**
* Parses the response from the site and returns the details of an anime.
*
* @param response the response from the site.
*/
protected abstract fun animeDetailsParse(response: Response): SAnime
/**
* Returns an observable with the updated episode list for an anime. Normally it's not needed to
* override this method. If an anime is licensed an empty episode list observable is returned
*
* @param anime the anime to look for episodes.
*/
override fun fetchEpisodeList(anime: SAnime): Observable<List<SEpisode>> {
return if (anime.status != SAnime.LICENSED) {
client.newCall(episodeListRequest(anime))
.asObservableSuccess()
.map { response ->
episodeListParse(response)
}
} else {
Observable.error(Exception("Licensed - No episodes to show"))
}
}
/**
* Returns the request for updating the episode list. Override only if it's needed to override
* the url, send different headers or request method like POST.
*
* @param anime the anime to look for episodes.
*/
protected open fun episodeListRequest(anime: SAnime): Request {
return GET(baseUrl + anime.url, headers)
}
/**
* Parses the response from the site and returns a list of episodes.
*
* @param response the response from the site.
*/
protected abstract fun episodeListParse(response: Response): List<SEpisode>
/**
* Returns an observable with the page list for a chapter.
*
* @param episode the episode whose video list has to be fetched.
*/
override fun fetchVideoList(episode: SEpisode): Observable<List<Video>> {
return client.newCall(videoListRequest(episode))
.asObservableSuccess()
.map { response ->
videoListParse(response).sort()
}
}
/**
* Returns the request for getting the episode link. Override only if it's needed to override
* the url, send different headers or request method like POST.
*
* @param episode the episode to look for links.
*/
protected open fun videoListRequest(episode: SEpisode): Request {
return GET(baseUrl + episode.url, headers)
}
/**
* Parses the response from the site and returns a list of pages.
*
* @param response the response from the site.
*/
protected abstract fun videoListParse(response: Response): List<Video>
/**
* Sorts the video list. Override this according to the user's preference.
*/
protected open fun List<Video>.sort(): List<Video> {
return this
}
/**
* Returns an observable with the page containing the source url of the image. If there's any
* error, it will return null instead of throwing an exception.
*
* @param video the video whose source image has to be fetched.
*/
open fun fetchVideoUrl(video: Video): Observable<String> {
return client.newCall(videoUrlRequest(video))
.asObservableSuccess()
.map { videoUrlParse(it) }
}
/**
* Returns the request for getting the url to the source image. Override only if it's needed to
* override the url, send different headers or request method like POST.
*
* @param video the chapter whose page list has to be fetched
*/
protected open fun videoUrlRequest(video: Video): Request {
return GET(video.url, headers)
}
/**
* Parses the response from the site and returns the absolute url to the source image.
*
* @param response the response from the site.
*/
protected abstract fun videoUrlParse(response: Response): String
/**
* Returns an observable with the response of the source image.
*
* @param video the page whose source image has to be downloaded.
*/
fun fetchVideo(video: Video): Observable<Response> {
val animeDownloadClient = client.newBuilder()
.callTimeout(30, TimeUnit.MINUTES)
.build()
return animeDownloadClient.newCachelessCallWithProgress(videoRequest(video, video.totalBytesDownloaded), video)
.asObservableSuccess()
}
/**
* Returns the request for getting the source image. Override only if it's needed to override
* the url, send different headers or request method like POST.
*
* @param video the video whose link has to be fetched
*/
protected open fun videoRequest(video: Video, bytes: Long = 0L): Request {
val headers = video.headers ?: headers
val newHeaders = if (bytes > 0L) {
Headers.Builder().addAll(headers).add("Range", "bytes=$bytes-").build()
} else {
null
}
return GET(video.videoUrl!!, newHeaders ?: headers)
}
/**
* Assigns the url of the episode without the scheme and domain. It saves some redundancy from
* database and the urls could still work after a domain change.
*
* @param url the full url to the episode.
*/
fun SEpisode.setUrlWithoutDomain(url: String) {
this.url = getUrlWithoutDomain(url)
}
/**
* Assigns the url of the anime without the scheme and domain. It saves some redundancy from
* database and the urls could still work after a domain change.
*
* @param url the full url to the anime.
*/
fun SAnime.setUrlWithoutDomain(url: String) {
this.url = getUrlWithoutDomain(url)
}
/**
* Returns the url of the given string without the scheme and domain.
*
* @param orig the full url.
*/
private fun getUrlWithoutDomain(orig: String): String {
return try {
val uri = URI(orig)
var out = uri.path
if (uri.query != null) {
out += "?" + uri.query
}
if (uri.fragment != null) {
out += "#" + uri.fragment
}
out
} catch (e: URISyntaxException) {
orig
}
}
/**
* Returns the url of the provided anime
*
* @since extensions-lib 14
* @param anime the anime
* @return url of the anime
*/
open fun getAnimeUrl(anime: SAnime): String {
return animeDetailsRequest(anime).url.toString()
}
/**
* Returns the url of the provided episode
*
* @since extensions-lib 14
* @param episode the episode
* @return url of the episode
*/
open fun getChapterUrl(episode: SEpisode): String {
return episode.url.toString()
}
/**
* Called before inserting a new episode into database. Use it if you need to override episode
* fields, like the title or the episode number. Do not change anything to [anime].
*
* @param episode the episode to be added.
* @param anime the anime of the episode.
*/
open fun prepareNewEpisode(episode: SEpisode, anime: SAnime) {}
/**
* Returns the list of filters for the source.
*/
override fun getFilterList() = AnimeFilterList()
}

25
app/src/main/java/eu/kanade/tachiyomi/animesource/online/AnimeHttpSourceFetcher.kt Normal file
View file

@ -0,0 +1,25 @@
package eu.kanade.tachiyomi.animesource.online
import eu.kanade.tachiyomi.animesource.model.Video
import rx.Observable
fun AnimeHttpSource.fetchUrlFromVideo(video: Video): Observable<Video> {
return Observable.just(video)
.filter { !it.videoUrl.isNullOrEmpty() }
.mergeWith(fetchRemainingVideoUrlsFromVideoList(video))
}
private fun AnimeHttpSource.fetchRemainingVideoUrlsFromVideoList(video: Video): Observable<Video> {
return Observable.just(video)
.filter { it.videoUrl.isNullOrEmpty() }
.concatMap { getVideoUrl(it) }
}
private fun AnimeHttpSource.getVideoUrl(video: Video): Observable<Video> {
video.status = Video.State.LOAD_VIDEO
return fetchVideoUrl(video)
.doOnError { video.status = Video.State.ERROR }
.onErrorReturn { null }
.doOnNext { video.videoUrl = it }
.map { video }
}

207
app/src/main/java/eu/kanade/tachiyomi/animesource/online/ParsedAnimeHttpSource.kt Normal file
View file

@ -0,0 +1,207 @@
package eu.kanade.tachiyomi.animesource.online
import eu.kanade.tachiyomi.animesource.model.AnimesPage
import eu.kanade.tachiyomi.animesource.model.SAnime
import eu.kanade.tachiyomi.animesource.model.SEpisode
import eu.kanade.tachiyomi.animesource.model.Video
import eu.kanade.tachiyomi.util.asJsoup
import okhttp3.Response
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
/**
* A simple implementation for sources from a website using Jsoup, an HTML parser.
*/
@Suppress("unused")
abstract class ParsedAnimeHttpSource : AnimeHttpSource() {
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
override fun popularAnimeParse(response: Response): AnimesPage {
val document = response.asJsoup()
val animes = document.select(popularAnimeSelector()).map { element ->
popularAnimeFromElement(element)
}
val hasNextPage = popularAnimeNextPageSelector()?.let { selector ->
document.select(selector).first()
} != null
return AnimesPage(animes, hasNextPage)
}
/**
* Returns the Jsoup selector that returns a list of [Element] corresponding to each anime.
*/
protected abstract fun popularAnimeSelector(): String
/**
* Returns an anime from the given [element]. Most sites only show the title and the url, it's
* totally fine to fill only those two values.
*
* @param element an element obtained from [popularAnimeSelector].
*/
protected abstract fun popularAnimeFromElement(element: Element): SAnime
/**
* Returns the Jsoup selector that returns the <a> tag linking to the next page, or null if
* there's no next page.
*/
protected abstract fun popularAnimeNextPageSelector(): String?
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
override fun searchAnimeParse(response: Response): AnimesPage {
val document = response.asJsoup()
val animes = document.select(searchAnimeSelector()).map { element ->
searchAnimeFromElement(element)
}
val hasNextPage = searchAnimeNextPageSelector()?.let { selector ->
document.select(selector).first()
} != null
return AnimesPage(animes, hasNextPage)
}
/**
* Returns the Jsoup selector that returns a list of [Element] corresponding to each anime.
*/
protected abstract fun searchAnimeSelector(): String
/**
* Returns an anime from the given [element]. Most sites only show the title and the url, it's
* totally fine to fill only those two values.
*
* @param element an element obtained from [searchAnimeSelector].
*/
protected abstract fun searchAnimeFromElement(element: Element): SAnime
/**
* Returns the Jsoup selector that returns the <a> tag linking to the next page, or null if
* there's no next page.
*/
protected abstract fun searchAnimeNextPageSelector(): String?
/**
* Parses the response from the site and returns a [AnimesPage] object.
*
* @param response the response from the site.
*/
override fun latestUpdatesParse(response: Response): AnimesPage {
val document = response.asJsoup()
val animes = document.select(latestUpdatesSelector()).map { element ->
latestUpdatesFromElement(element)
}
val hasNextPage = latestUpdatesNextPageSelector()?.let { selector ->
document.select(selector).first()
} != null
return AnimesPage(animes, hasNextPage)
}
/**
* Returns the Jsoup selector that returns a list of [Element] corresponding to each anime.
*/
protected abstract fun latestUpdatesSelector(): String
/**
* Returns an anime from the given [element]. Most sites only show the title and the url, it's
* totally fine to fill only those two values.
*
* @param element an element obtained from [latestUpdatesSelector].
*/
protected abstract fun latestUpdatesFromElement(element: Element): SAnime
/**
* Returns the Jsoup selector that returns the <a> tag linking to the next page, or null if
* there's no next page.
*/
protected abstract fun latestUpdatesNextPageSelector(): String?
/**
* Parses the response from the site and returns the details of an anime.
*
* @param response the response from the site.
*/
override fun animeDetailsParse(response: Response): SAnime {
return animeDetailsParse(response.asJsoup())
}
/**
* Returns the details of the anime from the given [document].
*
* @param document the parsed document.
*/
protected abstract fun animeDetailsParse(document: Document): SAnime
/**
* Parses the response from the site and returns a list of episodes.
*
* @param response the response from the site.
*/
override fun episodeListParse(response: Response): List<SEpisode> {
val document = response.asJsoup()
return document.select(episodeListSelector()).map { episodeFromElement(it) }
}
/**
* Returns the Jsoup selector that returns a list of [Element] corresponding to each episode.
*/
protected abstract fun episodeListSelector(): String
/**
* Returns an episode from the given element.
*
* @param element an element obtained from [episodeListSelector].
*/
protected abstract fun episodeFromElement(element: Element): SEpisode
/**
* Parses the response from the site and returns the page list.
*
* @param response the response from the site.
*/
override fun videoListParse(response: Response): List<Video> {
val document = response.asJsoup()
return document.select(videoListSelector()).map { videoFromElement(it) }
}
/**
* Returns the Jsoup selector that returns a list of [Element] corresponding to each video.
*/
protected abstract fun videoListSelector(): String
/**
* Returns a video from the given element.
*
* @param element an element obtained from [videoListSelector].
*/
protected abstract fun videoFromElement(element: Element): Video
/**
* Parse the response from the site and returns the absolute url to the source video.
*
* @param response the response from the site.
*/
override fun videoUrlParse(response: Response): String {
return videoUrlParse(response.asJsoup())
}
/**
* Returns the absolute url to the source image from the document.
*
* @param document the parsed document.
*/
protected abstract fun videoUrlParse(document: Document): String
}

406
app/src/main/java/eu/kanade/tachiyomi/source/online/HttpSource.kt Normal file
View file

@ -0,0 +1,406 @@
package eu.kanade.tachiyomi.source.online
import eu.kanade.tachiyomi.network.GET
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.asObservableSuccess
import eu.kanade.tachiyomi.network.awaitSuccess
import eu.kanade.tachiyomi.network.newCachelessCallWithProgress
import ani.dantotsu.aniyomi.source.CatalogueSource
import ani.dantotsu.aniyomi.source.model.FilterList
import ani.dantotsu.aniyomi.source.model.MangasPage
import ani.dantotsu.aniyomi.source.model.Page
import ani.dantotsu.aniyomi.source.model.SChapter
import ani.dantotsu.aniyomi.source.model.SManga
import okhttp3.Headers
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response
import rx.Observable
import uy.kohesive.injekt.injectLazy
import java.net.URI
import java.net.URISyntaxException
import java.security.MessageDigest
/**
* A simple implementation for sources from a website.
*/
abstract class HttpSource : CatalogueSource {
/**
* Network service.
*/
protected val network: NetworkHelper by injectLazy()
/**
* Base url of the website without the trailing slash, like: http://mysite.com
*/
abstract val baseUrl: String
/**
* Version id used to generate the source id. If the site completely changes and urls are
* incompatible, you may increase this value and it'll be considered as a new source.
*/
open val versionId = 1
/**
* Id of the source. By default it uses a generated id using the first 16 characters (64 bits)
* of the MD5 of the string: sourcename/language/versionId
* Note the generated id sets the sign bit to 0.
*/
override val id by lazy {
val key = "${name.lowercase()}/$lang/$versionId"
val bytes = MessageDigest.getInstance("MD5").digest(key.toByteArray())
(0..7).map { bytes[it].toLong() and 0xff shl 8 * (7 - it) }.reduce(Long::or) and Long.MAX_VALUE
}
/**
* Headers used for requests.
*/
val headers: Headers by lazy { headersBuilder().build() }
/**
* Default network client for doing requests.
*/
open val client: OkHttpClient
get() = network.client
/**
* Headers builder for requests. Implementations can override this method for custom headers.
*/
protected open fun headersBuilder() = Headers.Builder().apply {
add("User-Agent", network.defaultUserAgentProvider())
}
/**
* Visible name of the source.
*/
override fun toString() = "$name (${lang.uppercase()})"
/**
* Returns an observable containing a page with a list of manga. Normally it's not needed to
* override this method.
*
* @param page the page number to retrieve.
*/
override fun fetchPopularManga(page: Int): Observable<MangasPage> {
return client.newCall(popularMangaRequest(page))
.asObservableSuccess()
.map { response ->
popularMangaParse(response)
}
}
/**
* Returns the request for the popular manga given the page.
*
* @param page the page number to retrieve.
*/
protected abstract fun popularMangaRequest(page: Int): Request
/**
* Parses the response from the site and returns a [MangasPage] object.
*
* @param response the response from the site.
*/
protected abstract fun popularMangaParse(response: Response): MangasPage
/**
* Returns an observable containing a page with a list of manga. Normally it's not needed to
* override this method.
*
* @param page the page number to retrieve.
* @param query the search query.
* @param filters the list of filters to apply.
*/
override fun fetchSearchManga(page: Int, query: String, filters: FilterList): Observable<MangasPage> {
return Observable.defer {
try {
client.newCall(searchMangaRequest(page, query, filters)).asObservableSuccess()
} catch (e: NoClassDefFoundError) {
// RxJava doesn't handle Errors, which tends to happen during global searches
// if an old extension using non-existent classes is still around
throw RuntimeException(e)
}
}
.map { response ->
searchMangaParse(response)
}
}
/**
* Returns the request for the search manga given the page.
*
* @param page the page number to retrieve.
* @param query the search query.
* @param filters the list of filters to apply.
*/
protected abstract fun searchMangaRequest(page: Int, query: String, filters: FilterList): Request
/**
* Parses the response from the site and returns a [MangasPage] object.
*
* @param response the response from the site.
*/
protected abstract fun searchMangaParse(response: Response): MangasPage
/**
* Returns an observable containing a page with a list of latest manga updates.
*
* @param page the page number to retrieve.
*/
override fun fetchLatestUpdates(page: Int): Observable<MangasPage> {
return client.newCall(latestUpdatesRequest(page))
.asObservableSuccess()
.map { response ->
latestUpdatesParse(response)
}
}
/**
* Returns the request for latest manga given the page.
*
* @param page the page number to retrieve.
*/
protected abstract fun latestUpdatesRequest(page: Int): Request
/**
* Parses the response from the site and returns a [MangasPage] object.
*
* @param response the response from the site.
*/
protected abstract fun latestUpdatesParse(response: Response): MangasPage
/**
* Returns an observable with the updated details for a manga. Normally it's not needed to
* override this method.
*
* @param manga the manga to be updated.
*/
override fun fetchMangaDetails(manga: SManga): Observable<SManga> {
return client.newCall(mangaDetailsRequest(manga))
.asObservableSuccess()
.map { response ->
mangaDetailsParse(response).apply { initialized = true }
}
}
/**
* Returns the request for the details of a manga. Override only if it's needed to change the
* url, send different headers or request method like POST.
*
* @param manga the manga to be updated.
*/
open fun mangaDetailsRequest(manga: SManga): Request {
return GET(baseUrl + manga.url, headers)
}
/**
* Parses the response from the site and returns the details of a manga.
*
* @param response the response from the site.
*/
protected abstract fun mangaDetailsParse(response: Response): SManga
/**
* Returns an observable with the updated chapter list for a manga. Normally it's not needed to
* override this method. If a manga is licensed an empty chapter list observable is returned
*
* @param manga the manga to look for chapters.
*/
override fun fetchChapterList(manga: SManga): Observable<List<SChapter>> {
return if (manga.status != SManga.LICENSED) {
client.newCall(chapterListRequest(manga))
.asObservableSuccess()
.map { response ->
chapterListParse(response)
}
} else {
Observable.error(Exception("Licensed - No chapters to show"))
}
}
/**
* Returns the request for updating the chapter list. Override only if it's needed to override
* the url, send different headers or request method like POST.
*
* @param manga the manga to look for chapters.
*/
protected open fun chapterListRequest(manga: SManga): Request {
return GET(baseUrl + manga.url, headers)
}
/**
* Parses the response from the site and returns a list of chapters.
*
* @param response the response from the site.
*/
protected abstract fun chapterListParse(response: Response): List<SChapter>
/**
* Returns an observable with the page list for a chapter.
*
* @param chapter the chapter whose page list has to be fetched.
*/
override fun fetchPageList(chapter: SChapter): Observable<List<Page>> {
return client.newCall(pageListRequest(chapter))
.asObservableSuccess()
.map { response ->
pageListParse(response)
}
}
/**
* Returns the request for getting the page list. Override only if it's needed to override the
* url, send different headers or request method like POST.
*
* @param chapter the chapter whose page list has to be fetched.
*/
protected open fun pageListRequest(chapter: SChapter): Request {
return GET(baseUrl + chapter.url, headers)
}
/**
* Parses the response from the site and returns a list of pages.
*
* @param response the response from the site.
*/
protected abstract fun pageListParse(response: Response): List<Page>
/**
* Returns an observable with the page containing the source url of the image. If there's any
* error, it will return null instead of throwing an exception.
*
* @param page the page whose source image has to be fetched.
*/
open fun fetchImageUrl(page: Page): Observable<String> {
return client.newCall(imageUrlRequest(page))
.asObservableSuccess()
.map { imageUrlParse(it) }
}
/**
* Returns the request for getting the url to the source image. Override only if it's needed to
* override the url, send different headers or request method like POST.
*
* @param page the chapter whose page list has to be fetched
*/
protected open fun imageUrlRequest(page: Page): Request {
return GET(page.url, headers)
}
/**
* Parses the response from the site and returns the absolute url to the source image.
*
* @param response the response from the site.
*/
protected abstract fun imageUrlParse(response: Response): String
/**
* Returns an observable with the response of the source image.
*
* @param page the page whose source image has to be downloaded.
*/
fun fetchImage(page: Page): Observable<Response> {
// images will be cached or saved manually, so don't take up network cache
return client.newCachelessCallWithProgress(imageRequest(page), page)
.asObservableSuccess()
}
/**
* Returns the response of the source image.
*
* @param page the page whose source image has to be downloaded.
*/
open suspend fun getImage(page: Page): Response {
// images will be cached or saved manually, so don't take up network cache
return client.newCachelessCallWithProgress(imageRequest(page), page)
.awaitSuccess()
}
/**
* Returns the request for getting the source image. Override only if it's needed to override
* the url, send different headers or request method like POST.
*
* @param page the chapter whose page list has to be fetched
*/
protected open fun imageRequest(page: Page): Request {
return GET(page.imageUrl!!, headers)
}
/**
* Assigns the url of the chapter without the scheme and domain. It saves some redundancy from
* database and the urls could still work after a domain change.
*
* @param url the full url to the chapter.
*/
fun SChapter.setUrlWithoutDomain(url: String) {
this.url = getUrlWithoutDomain(url)
}
/**
* Assigns the url of the manga without the scheme and domain. It saves some redundancy from
* database and the urls could still work after a domain change.
*
* @param url the full url to the manga.
*/
fun SManga.setUrlWithoutDomain(url: String) {
this.url = getUrlWithoutDomain(url)
}
/**
* Returns the url of the given string without the scheme and domain.
*
* @param orig the full url.
*/
private fun getUrlWithoutDomain(orig: String): String {
return try {
val uri = URI(orig.replace(" ", "%20"))
var out = uri.path
if (uri.query != null) {
out += "?" + uri.query
}
if (uri.fragment != null) {
out += "#" + uri.fragment
}
out
} catch (e: URISyntaxException) {
orig
}
}
/**
* Returns the url of the provided manga
*
* @since extensions-lib 1.4
* @param manga the manga
* @return url of the manga
*/
open fun getMangaUrl(manga: SManga): String {
return mangaDetailsRequest(manga).url.toString()
}
/**
* Returns the url of the provided chapter
*
* @since extensions-lib 1.4
* @param chapter the chapter
* @return url of the chapter
*/
open fun getChapterUrl(chapter: SChapter): String {
return pageListRequest(chapter).url.toString()
}
/**
* Called before inserting a new chapter into database. Use it if you need to override chapter
* fields, like the title or the chapter number. Do not change anything to [manga].
*
* @param chapter the chapter to be added.
* @param manga the manga of the chapter.
*/
open fun prepareNewChapter(chapter: SChapter, manga: SManga) {}
/**
* Returns the list of filters for the source.
*/
override fun getFilterList() = FilterList()
}