Release 0.20.1 (955)

[YouTube] Fix search for some users
[YouTube] Fix random decryption exceptions
[SoundCloud] URLs that end with a slash are now parsed correctly

.github/CONTRIBUTING.md

@@ -12,57 +12,54 @@ add a comment to it. You'll see exactly what is sent, the system is 100% transpa
 ## Issue reporting/feature requests
 
 * Search the [existing issues](https://github.com/TeamNewPipe/NewPipe/issues) first to make sure your issue/feature
-hasn't been reported/requested before
-* Check whether your issue/feature is already fixed/implemented
-* Check if the issue still exists in the latest release/beta version
-* If you are an Android/Java developer, you are always welcome to fix/implement an issue/a feature yourself. PRs welcome!
+hasn't been reported/requested before.
+* Check whether your issue/feature is already fixed/implemented.
+* Check if the issue still exists in the latest release/beta version.
+* If you are an Android/Java developer, you are always welcome to fix an issue or implement a feature yourself. PRs welcome!
 * We use English for development. Issues in other languages will be closed and ignored.
 * Please only add *one* issue at a time. Do not put multiple issues into one thread.
-* When reporting a bug please give us a context, and a description how to reproduce it.
-* Issues that only contain a generated bug report, but no description might be closed.
+* Follow the template! Issues or feature requests not matching the template might be closed.
 
 ## Bug Fixing
 * If you want to help NewPipe to become free of bugs (this is our utopic goal for NewPipe), you can send us an email to
-tnp@newpipe.schabi.org to let me know that you intend to help. We'll send you further instructions. You may, on request,
-register at our [Sentry](https://sentry.schabi.org) instance (see section "Crash reporting" for more information.
+<a href="mailto:tnp@newpipe.schabi.org">tnp@newpipe.schabi.org</a> to let us know that you intend to help. We'll send you further instructions. You may, on request,
+register at our [Sentry](https://sentry.schabi.org) instance (see section "Crash reporting" for more information).
 
 ## Translation
 
-* NewPipe can be translated via [Weblate](https://hosted.weblate.org/projects/newpipe/strings/). You can log in there
+* NewPipe is translated via [Weblate](https://hosted.weblate.org/projects/newpipe/strings/). You can log in there
 with your GitHub account.
+* If the language you want to translate is not on Weblate, you can add it: see [How to add a new language](https://github.com/TeamNewPipe/NewPipe/wiki/How-to-add-a-new-language-to-NewPipe) in the wiki.
 
 ## Code contribution
 
-* Stick to NewPipe's style conventions (well, just look the other code and then do it the same way :))
-* Do not bring non-free software (e.g., binary blobs) into the project. Also, make sure you do not introduce Google
+* If you want to add a feature or change one, please open an issue describing your change. This gives the team and community a chance to give feedback before you spend any time on something that could be done differently or not done at all. It also prevents two contributors from working on the same thing and one being disappointed when only one user's code can be added.
+* Stick to NewPipe's style conventions: follow [checkStyle](https://github.com/checkstyle/checkstyle). It will run each time you build the project.
+* Do not bring non-free software (e.g. binary blobs) into the project. Also, make sure you do not introduce Google
   libraries.
-* Stick to [F-Droid contribution guidelines](https://f-droid.org/wiki/page/Inclusion_Policy)
-* Make changes on a separate branch, not on the master branch. This is commonly known as *feature branch workflow*. You
-  may then send your changes as a pull request on GitHub. Patches to the email address mentioned in this document might
-  not be considered, GitHub is the primary platform. (This only affects you if you are a member of TeamNewPipe)
+* Stick to [F-Droid contribution guidelines](https://f-droid.org/wiki/page/Inclusion_Policy).
+* Make changes on a separate branch with a meaningful name, not on the master neither dev branch. This is commonly known as *feature branch workflow*. You
+  may then send your changes as a pull request (PR) on GitHub.
 * When submitting changes, you confirm that your code is licensed under the terms of the
   [GNU General Public License v3](https://www.gnu.org/licenses/gpl-3.0.html).
 * Please test (compile and run) your code before you submit changes! Ideally, provide test feedback in the PR
   description. Untested code will **not** be merged!
 * Try to figure out yourself why builds on our CI fail.
 * Make sure your PR is up-to-date with the rest of the code. Often, a simple click on "Update branch" will do the job,
-  but if not, you are asked to merge the master branch manually and resolve the problems on your own. That will make the
+  but if not, you are asked to rebase the dev branch manually and resolve the problems on your own. You can find help [on the wiki](https://github.com/TeamNewPipe/NewPipe/wiki/How-to-merge-a-PR). That will make the
   maintainers' jobs way easier.
 * Please show intention to maintain your features and code after you contributed it. Unmaintained code is a hassle for
   the core developers, and just adds work. If you do not intend to maintain features you contributed, please think again
   about submission, or clearly state that in the description of your PR.
 * Respond yourselves if someone requests changes or otherwise raises issues about your PRs.
-* Check if your contributions align with the [fdroid inclusion guidelines](https://f-droid.org/en/docs/Inclusion_Policy/).
-* Check if your submission can be build with the current fdroid build server setup.
 * Send PR that only cover one specific issue/solution/bug. Do not send PRs that are huge and consists of multiple
   independent solutions.
 
 ## Communication
 
-* WE DO NOW HAVE A MAILING LIST: [newpipe@list.schabi.org](https://list.schabi.org/cgi-bin/mailman/listinfo/newpipe).
 * There is an IRC channel on Freenode which is regularly visited by the core team and other developers:
   [#newpipe](irc:irc.freenode.net/newpipe). [Click here for Webchat](https://webchat.freenode.net/?channels=newpipe)!
 * If you want to get in touch with the core team or one of our other contributors you can send an email to
-  tnp(at)schabi.org. Please do not send issue reports, they will be ignored and remain unanswered! Use the GitHub issue
+  <a href="mailto:tnp@newpipe.schabi.org">tnp@newpipe.schabi.org</a>. Please do not send issue reports, they will be ignored and remain unanswered! Use the GitHub issue
   tracker described above!
-* Feel free to post suggestions, changes, ideas etc. on GitHub, IRC or the mailing list!
+* Feel free to post suggestions, changes, ideas etc. on GitHub or IRC!

.github/ISSUE_TEMPLATE.md

@@ -1,3 +0,0 @@
-- [ ] I carefully read the [contribution guidelines](https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md) and agree to them.
-- [ ] I checked if the issue/feature exists in the latest version.
-- [ ] I did use the [incredible bugreport to markdown converter](https://teamnewpipe.github.io/CrashReportToMarkdown/) to paste bug reports.

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,46 @@
+---
+name: Bug report
+about: Create a bug report to help us improve
+labels: bug
+assignees: ''
+
+---
+
+<!--
+Oh no, a bug! It happens. Thanks for reporting an issue with NewPipe. If this is your first bug report, read the following information before proceeding:
+
+Please note, we only support the latest version of NewPipe. In order to check your app version, open the left drawer and click on "About". If you don't have the latest version, upgrade to it and reproduce the problem before opening the issue. The release page (https://github.com/TeamNewPipe/NewPipe/releases/latest) is where you can get it.
+
+P.S.: Our contribution guidelines might be a nice document to read before you fill out the report :) You can find it at https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md
+
+To make it easier for us to help you please enter detailed information in the template we have provided below. If a section isn't relevant, just delete it, though it would be helpful to still provide as much detail as possible.
+-->
+
+<!-- The comments between these brackets won't show up in the submitted issue (as you can see in the preview). -->
+
+### Version
+<!-- Which version are you using? Hopefully the latest! We just told you that above! -->
+-
+
+### Steps to reproduce the bug
+<!--
+1. Go to '...'
+2. Press on '....'
+3. Swipe down to '....'
+-->
+
+<!-- If you can't cause the bug to show up again reliably (and hence don't have a proper set of steps to give us), please still try to give as many details as possible on how you think you encountered the bug. -->
+
+### Expected behavior
+<!-- Tell us what you expect to happen. -->
+
+### Actual behaviour
+<!-- Tell us what happens instead. -->
+
+### Screenshots/Screen recordings
+<!-- If applicable, add screenshots or a screen recording to help explain your problem. GitHub supports uploading them directly in the issue text box. If your file is too big for Github to accept, feel free to paste a link from an image/video hoster here instead. -->
+
+### Logs
+<!-- If your bug includes a crash (where you're shown the Error Report page with a bunch of info), copy it to the clipboard (there is a share button for this), head over to our bug report to markdown converter at https://teamnewpipe.github.io/CrashReportToMarkdown/ and paste it. Copy the converted text (it is MUCH easier to read this way) from there and paste it here: -->
+
+<!-- That's right, here! -->

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,39 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+labels: enhancement
+assignees: ''
+
+---
+<!-- Hey. Our contribution guidelines (https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md) might be an appropriate
+ document to read before you fill out the request :) -->
+
+<!-- The comments between these brackets won't show up in the submitted issue (as you can see in the preview). -->
+
+#### Describe the feature you want
+<!-- A clear and concise description of what you want to happen. PLEASE MAKE SURE it is one feature ONLY. You should open separate issues for separate feature requests, because those issues will be used to track their status.
+Example: *I think it would be nice if you add feature Y which makes X possible.*
+
+Optionally, also describe alternatives you've considered.
+Example: *Z is also a good alternative. Not as good as Y, but at least...* or *I considered Z, but that didn't turn out to be a good idea because...* -->
+
+<!-- Write below this -->
+
+#### Is your feature request related to a problem? Please describe it
+<!-- A clear and concise description of what the problem is. Maybe the developers could brainstorm and come up with a better solution to your problem. If they exist, link to related Issues and/or PRs for developers to keep track easier.
+Example: *I want to do X, but there is no way to do it.* -->
+
+<!-- Write below this -->
+
+#### Additional context
+<!-- Add any other context, like screenshots, about the feature request here.
+Example: *Here's a photo of my cat!* -->
+
+<!-- Write below this -->
+
+#### How will you/everyone benefit from this feature?
+<!-- Convince us! How does it change your NewPipe experience and/or your life?
+The better this paragraph is, the more likely a developer will think about working on it.
+Example: *This feature will help us colonize the galaxy! -->
+
+<!-- Write below this -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -1 +1,28 @@
+<!-- Hey there. Thank you so much for improving NewPipe. Please take a moment to fill out the following suggestion on how to structure this PR description. Having roughly the same layout helps everyone considerably :)-->
+
+#### What is it?
+- [ ] Bug fix (user facing)
+- [ ] Feature (user facing)
+- [ ] Code base improvement (dev facing)
+- [ ] Meta improvement to the project (dev facing)
+
+#### Description of the changes in your PR
+<!-- While bullet points are the norm in this section, feel free to write a text instead if you can't fit it in a list -->
+- record videos
+- create clones
+- take over the world
+
+#### Fixes the following issue(s)
+<!-- Also add reddit or other links which are relevant to your change. -->
+- 
+
+#### Relies on the following changes
+<!-- Delete this if it doesn't apply to you. -->
+- 
+
+#### Testing apk
+<!-- Ensure that you have your changes on a new branch which has a meaningful name. This name will be used as a suffix for the app ID to allow installing and testing multiple versions of NewPipe. Do NOT name your branches like "patch-0" and "feature-1". For example, if your PR implements a bug fix for comments, an appropriate branch name would be "commentfix". -->
+debug.zip
+
+#### Agreement
 - [ ] I carefully read the [contribution guidelines](https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md) and agree to them.

.gitignore

@@ -10,3 +10,11 @@
 *~
 .weblate
 *.class
+
+# vscode / eclipse files
+*.classpath
+*.project
+*.settings
+bin/
+.vscode/
+*.code-workspace

.travis.yml

@@ -5,13 +5,13 @@ android:
   components:
     # The BuildTools version used by NewPipe
     - tools
-    - build-tools-28.0.3
+    - build-tools-29.0.3
 
     # The SDK version used to compile NewPipe
-    - android-28
+    - android-29
 
 before_install: 
-  - yes | sdkmanager "platforms;android-28"
+  - yes | sdkmanager "platforms;android-29"
 script: ./gradlew -Dorg.gradle.jvmargs=-Xmx1536m assembleDebug lintDebug testDebugUnitTest
 
 licenses:

README.md

@@ -4,10 +4,10 @@
 <p align="center"><a href="https://f-droid.org/packages/org.schabi.newpipe/"><img src="https://f-droid.org/wiki/images/0/06/F-Droid-button_get-it-on.png"></a></p> 
 
 <p align="center">
-<a href="https://github.com/TeamNewPipe/NewPipe" alt="GitHub release"><img src="https://img.shields.io/github/release/TeamNewPipe/NewPipe.svg" ></a>
+<a href="https://github.com/TeamNewPipe/NewPipe/releases" alt="GitHub release"><img src="https://img.shields.io/github/release/TeamNewPipe/NewPipe.svg" ></a>
 <a href="https://www.gnu.org/licenses/gpl-3.0" alt="License: GPLv3"><img src="https://img.shields.io/badge/License-GPL%20v3-blue.svg"></a>
 <a href="https://travis-ci.org/TeamNewPipe/NewPipe" alt="Build Status"><img src="https://travis-ci.org/TeamNewPipe/NewPipe.svg"></a>
-<a href="https://hosted.weblate.org/engage/NewPipe/" alt="Translation Status"><img src="https://hosted.weblate.org/widgets/NewPipe/-/svg-badge.svg"></a>
+<a href="https://hosted.weblate.org/engage/newpipe/" alt="Translation Status"><img src="https://hosted.weblate.org/widgets/newpipe/-/svg-badge.svg"></a>
 <a href="http://webchat.freenode.net/?channels=%23newpipe" alt="IRC channel: #newpipe"><img src="https://img.shields.io/badge/IRC%20chat-%23newpipe-brightgreen.svg"></a>
 <a href="https://www.bountysource.com/teams/newpipe" alt="Bountysource bounties"><img src="https://img.shields.io/bountysource/team/newpipe/activity.svg?colorB=cd201f"></a>
 </p>

app/src/androidTest/java/org/schabi/newpipe/database/AppDatabaseTest.kt

@@ -0,0 +1,119 @@
+package org.schabi.newpipe.database
+
+import android.content.ContentValues
+import android.database.sqlite.SQLiteDatabase
+import androidx.room.Room
+import androidx.room.testing.MigrationTestHelper
+import androidx.sqlite.db.framework.FrameworkSQLiteOpenHelperFactory
+import androidx.test.core.app.ApplicationProvider
+import androidx.test.ext.junit.runners.AndroidJUnit4
+import androidx.test.platform.app.InstrumentationRegistry
+import org.junit.Assert.assertEquals
+import org.junit.Assert.assertNull
+import org.junit.Rule
+import org.junit.Test
+import org.junit.runner.RunWith
+import org.schabi.newpipe.extractor.stream.StreamType
+
+@RunWith(AndroidJUnit4::class)
+class AppDatabaseTest {
+    companion object {
+        private const val DEFAULT_SERVICE_ID = 0
+        private const val DEFAULT_URL = "https://www.youtube.com/watch?v=cDphUib5iG4"
+        private const val DEFAULT_TITLE = "Test Title"
+        private val DEFAULT_TYPE = StreamType.VIDEO_STREAM
+        private const val DEFAULT_DURATION = 480L
+        private const val DEFAULT_UPLOADER_NAME = "Uploader Test"
+        private const val DEFAULT_THUMBNAIL = "https://example.com/example.jpg"
+
+        private const val DEFAULT_SECOND_SERVICE_ID = 0
+        private const val DEFAULT_SECOND_URL = "https://www.youtube.com/watch?v=ncQU6iBn5Fc"
+    }
+
+    @get:Rule
+    val testHelper = MigrationTestHelper(InstrumentationRegistry.getInstrumentation(),
+            AppDatabase::class.java.canonicalName, FrameworkSQLiteOpenHelperFactory())
+
+    @Test
+    fun migrateDatabaseFrom2to3() {
+        val databaseInV2 = testHelper.createDatabase(AppDatabase.DATABASE_NAME, Migrations.DB_VER_2)
+
+        databaseInV2.run {
+            insert("streams", SQLiteDatabase.CONFLICT_FAIL, ContentValues().apply {
+                // put("uid", null)
+                put("service_id", DEFAULT_SERVICE_ID)
+                put("url", DEFAULT_URL)
+                put("title", DEFAULT_TITLE)
+                put("stream_type", DEFAULT_TYPE.name)
+                put("duration", DEFAULT_DURATION)
+                put("uploader", DEFAULT_UPLOADER_NAME)
+                put("thumbnail_url", DEFAULT_THUMBNAIL)
+            })
+            insert("streams", SQLiteDatabase.CONFLICT_FAIL, ContentValues().apply {
+                // put("uid", null)
+                put("service_id", DEFAULT_SECOND_SERVICE_ID)
+                put("url", DEFAULT_SECOND_URL)
+                // put("title", null)
+                // put("stream_type", null)
+                // put("duration", null)
+                // put("uploader", null)
+                // put("thumbnail_url", null)
+            })
+            insert("streams", SQLiteDatabase.CONFLICT_FAIL, ContentValues().apply {
+                // put("uid", null)
+                put("service_id", DEFAULT_SERVICE_ID)
+                // put("url", null)
+                // put("title", null)
+                // put("stream_type", null)
+                // put("duration", null)
+                // put("uploader", null)
+                // put("thumbnail_url", null)
+            })
+            close()
+        }
+
+        testHelper.runMigrationsAndValidate(AppDatabase.DATABASE_NAME, Migrations.DB_VER_3,
+                true, Migrations.MIGRATION_2_3)
+
+        val migratedDatabaseV3 = getMigratedDatabase()
+        val listFromDB = migratedDatabaseV3.streamDAO().all.blockingFirst()
+
+        // Only expect 2, the one with the null url will be ignored
+        assertEquals(2, listFromDB.size)
+
+        val streamFromMigratedDatabase = listFromDB[0]
+        assertEquals(DEFAULT_SERVICE_ID, streamFromMigratedDatabase.serviceId)
+        assertEquals(DEFAULT_URL, streamFromMigratedDatabase.url)
+        assertEquals(DEFAULT_TITLE, streamFromMigratedDatabase.title)
+        assertEquals(DEFAULT_TYPE, streamFromMigratedDatabase.streamType)
+        assertEquals(DEFAULT_DURATION, streamFromMigratedDatabase.duration)
+        assertEquals(DEFAULT_UPLOADER_NAME, streamFromMigratedDatabase.uploader)
+        assertEquals(DEFAULT_THUMBNAIL, streamFromMigratedDatabase.thumbnailUrl)
+        assertNull(streamFromMigratedDatabase.viewCount)
+        assertNull(streamFromMigratedDatabase.textualUploadDate)
+        assertNull(streamFromMigratedDatabase.uploadDate)
+        assertNull(streamFromMigratedDatabase.isUploadDateApproximation)
+
+        val secondStreamFromMigratedDatabase = listFromDB[1]
+        assertEquals(DEFAULT_SECOND_SERVICE_ID, secondStreamFromMigratedDatabase.serviceId)
+        assertEquals(DEFAULT_SECOND_URL, secondStreamFromMigratedDatabase.url)
+        assertEquals("", secondStreamFromMigratedDatabase.title)
+        // Should fallback to VIDEO_STREAM
+        assertEquals(StreamType.VIDEO_STREAM, secondStreamFromMigratedDatabase.streamType)
+        assertEquals(0, secondStreamFromMigratedDatabase.duration)
+        assertEquals("", secondStreamFromMigratedDatabase.uploader)
+        assertEquals("", secondStreamFromMigratedDatabase.thumbnailUrl)
+        assertNull(secondStreamFromMigratedDatabase.viewCount)
+        assertNull(secondStreamFromMigratedDatabase.textualUploadDate)
+        assertNull(secondStreamFromMigratedDatabase.uploadDate)
+        assertNull(secondStreamFromMigratedDatabase.isUploadDateApproximation)
+    }
+
+    private fun getMigratedDatabase(): AppDatabase {
+        val database: AppDatabase = Room.databaseBuilder(ApplicationProvider.getApplicationContext(),
+                AppDatabase::class.java, AppDatabase.DATABASE_NAME)
+                .build()
+        testHelper.closeWhenFinished(database)
+        return database
+    }
+}

app/src/androidTest/java/org/schabi/newpipe/report/ErrorInfoTest.java

@@ -1,8 +1,9 @@
 package org.schabi.newpipe.report;
 
 import android.os.Parcel;
-import androidx.test.filters.LargeTest;
+
 import androidx.test.ext.junit.runners.AndroidJUnit4;
+import androidx.test.filters.LargeTest;
 
 import org.junit.Test;
 import org.junit.runner.RunWith;
@@ -12,20 +13,21 @@ import org.schabi.newpipe.report.ErrorActivity.ErrorInfo;
 import static org.junit.Assert.assertEquals;
 
 /**
- * Instrumented tests for {@link ErrorInfo}
+ * Instrumented tests for {@link ErrorInfo}.
  */
 @RunWith(AndroidJUnit4.class)
 @LargeTest
 public class ErrorInfoTest {
 
     @Test
-    public void errorInfo_testParcelable() {
-        ErrorInfo info = ErrorInfo.make(UserAction.USER_REPORT, "youtube", "request", R.string.general_error);
+    public void errorInfoTestParcelable() {
+        final ErrorInfo info = ErrorInfo.make(UserAction.USER_REPORT, "youtube", "request",
+                R.string.general_error);
         // Obtain a Parcel object and write the parcelable object to it:
-        Parcel parcel = Parcel.obtain();
+        final Parcel parcel = Parcel.obtain();
         info.writeToParcel(parcel, 0);
         parcel.setDataPosition(0);
-        ErrorInfo infoFromParcel = ErrorInfo.CREATOR.createFromParcel(parcel);
+        final ErrorInfo infoFromParcel = ErrorInfo.CREATOR.createFromParcel(parcel);
 
         assertEquals(UserAction.USER_REPORT, infoFromParcel.userAction);
         assertEquals("youtube", infoFromParcel.serviceName);
@@ -34,4 +36,4 @@ public class ErrorInfoTest {
 
         parcel.recycle();
     }
-}
+}

app/src/debug/AndroidManifest.xml

@@ -6,12 +6,5 @@
 
     <application
         android:name=".DebugApp"
-        android:label="NewPipe Debug"
-        tools:replace="android:name, android:label">
-        <activity
-            android:name=".MainActivity"
-            android:label="NewPipe Debug"
-            tools:replace="android:label"/>
-    </application>
-
+        tools:replace="android:name" />
 </manifest>

app/src/debug/java/org/schabi/newpipe/DebugApp.java

@@ -1,104 +0,0 @@
-package org.schabi.newpipe;
-
-import android.content.Context;
-import android.content.SharedPreferences;
-import android.preference.PreferenceManager;
-import androidx.annotation.NonNull;
-import androidx.multidex.MultiDex;
-
-import com.facebook.stetho.Stetho;
-import com.facebook.stetho.okhttp3.StethoInterceptor;
-import com.squareup.leakcanary.AndroidHeapDumper;
-import com.squareup.leakcanary.DefaultLeakDirectoryProvider;
-import com.squareup.leakcanary.HeapDumper;
-import com.squareup.leakcanary.LeakCanary;
-import com.squareup.leakcanary.LeakDirectoryProvider;
-import com.squareup.leakcanary.RefWatcher;
-
-import org.schabi.newpipe.extractor.downloader.Downloader;
-
-import java.io.File;
-import java.util.concurrent.TimeUnit;
-
-import okhttp3.OkHttpClient;
-
-public class DebugApp extends App {
-    private static final String TAG = DebugApp.class.toString();
-
-    @Override
-    protected void attachBaseContext(Context base) {
-        super.attachBaseContext(base);
-        MultiDex.install(this);
-    }
-
-    @Override
-    public void onCreate() {
-        super.onCreate();
-        initStetho();
-    }
-
-    @Override
-    protected Downloader getDownloader() {
-        return DownloaderImpl.init(new OkHttpClient.Builder()
-                .addNetworkInterceptor(new StethoInterceptor()));
-    }
-
-    private void initStetho() {
-        // Create an InitializerBuilder
-        Stetho.InitializerBuilder initializerBuilder =
-                Stetho.newInitializerBuilder(this);
-
-        // Enable Chrome DevTools
-        initializerBuilder.enableWebKitInspector(
-                Stetho.defaultInspectorModulesProvider(this)
-        );
-
-        // Enable command line interface
-        initializerBuilder.enableDumpapp(
-                Stetho.defaultDumperPluginsProvider(getApplicationContext())
-        );
-
-        // Use the InitializerBuilder to generate an Initializer
-        Stetho.Initializer initializer = initializerBuilder.build();
-
-        // Initialize Stetho with the Initializer
-        Stetho.initialize(initializer);
-    }
-
-    @Override
-    protected boolean isDisposedRxExceptionsReported() {
-        return PreferenceManager.getDefaultSharedPreferences(this)
-                .getBoolean(getString(R.string.allow_disposed_exceptions_key), false);
-    }
-
-    @Override
-    protected RefWatcher installLeakCanary() {
-        return LeakCanary.refWatcher(this)
-                .heapDumper(new ToggleableHeapDumper(this))
-                // give each object 10 seconds to be gc'ed, before leak canary gets nosy on it
-                .watchDelay(10, TimeUnit.SECONDS)
-                .buildAndInstall();
-    }
-
-    public static class ToggleableHeapDumper implements HeapDumper {
-        private final HeapDumper dumper;
-        private final SharedPreferences preferences;
-        private final String dumpingAllowanceKey;
-
-        ToggleableHeapDumper(@NonNull final Context context) {
-            LeakDirectoryProvider leakDirectoryProvider = new DefaultLeakDirectoryProvider(context);
-            this.dumper = new AndroidHeapDumper(context, leakDirectoryProvider);
-            this.preferences = PreferenceManager.getDefaultSharedPreferences(context);
-            this.dumpingAllowanceKey = context.getString(R.string.allow_heap_dumping_key);
-        }
-
-        private boolean isDumpingAllowed() {
-            return preferences.getBoolean(dumpingAllowanceKey, false);
-        }
-
-        @Override
-        public File dumpHeap() {
-            return isDumpingAllowed() ? dumper.dumpHeap() : HeapDumper.RETRY_LATER;
-        }
-    }
-}

app/src/debug/java/org/schabi/newpipe/DebugApp.kt

@@ -0,0 +1,59 @@
+package org.schabi.newpipe
+
+import androidx.multidex.MultiDex
+import androidx.preference.PreferenceManager
+import com.facebook.stetho.Stetho
+import com.facebook.stetho.okhttp3.StethoInterceptor
+import leakcanary.AppWatcher
+import leakcanary.LeakCanary
+import okhttp3.OkHttpClient
+import org.schabi.newpipe.extractor.downloader.Downloader
+
+class DebugApp : App() {
+    override fun onCreate() {
+        super.onCreate()
+        initStetho()
+
+        // Give each object 10 seconds to be GC'ed, before LeakCanary gets nosy on it
+        AppWatcher.config = AppWatcher.config.copy(watchDurationMillis = 10000)
+        LeakCanary.config = LeakCanary.config.copy(dumpHeap = PreferenceManager
+                .getDefaultSharedPreferences(this).getBoolean(getString(
+                        R.string.allow_heap_dumping_key), false))
+    }
+
+    override fun getDownloader(): Downloader {
+        val downloader = DownloaderImpl.init(OkHttpClient.Builder()
+                .addNetworkInterceptor(StethoInterceptor()))
+        setCookiesToDownloader(downloader)
+        return downloader
+    }
+
+    override fun initACRA() {
+        // install MultiDex before initializing ACRA
+        MultiDex.install(this)
+        super.initACRA()
+    }
+
+    private fun initStetho() {
+        // Create an InitializerBuilder
+        val initializerBuilder = Stetho.newInitializerBuilder(this)
+
+        // Enable Chrome DevTools
+        initializerBuilder.enableWebKitInspector(Stetho.defaultInspectorModulesProvider(this))
+
+        // Enable command line interface
+        initializerBuilder.enableDumpapp(
+                Stetho.defaultDumperPluginsProvider(applicationContext))
+
+        // Use the InitializerBuilder to generate an Initializer
+        val initializer = initializerBuilder.build()
+
+        // Initialize Stetho with the Initializer
+        Stetho.initialize(initializer)
+    }
+
+    override fun isDisposedRxExceptionsReported(): Boolean {
+        return PreferenceManager.getDefaultSharedPreferences(this)
+                .getBoolean(getString(R.string.allow_disposed_exceptions_key), false)
+    }
+}

app/src/main/java/com/google/android/material/appbar/FlingBehavior.java

@@ -1,24 +1,80 @@
 package com.google.android.material.appbar;
 
 import android.content.Context;
+import android.graphics.Rect;
 import android.util.AttributeSet;
 import android.view.MotionEvent;
+import android.view.View;
 import android.widget.OverScroller;
 
+import androidx.annotation.NonNull;
 import androidx.annotation.Nullable;
 import androidx.coordinatorlayout.widget.CoordinatorLayout;
+import org.schabi.newpipe.R;
 
 import java.lang.reflect.Field;
+import java.util.Arrays;
+import java.util.List;
 
-// check this https://stackoverflow.com/questions/56849221/recyclerview-fling-causes-laggy-while-appbarlayout-is-scrolling/57997489#57997489
+// See https://stackoverflow.com/questions/56849221#57997489
 public final class FlingBehavior extends AppBarLayout.Behavior {
+    private final Rect focusScrollRect = new Rect();
 
-    public FlingBehavior(Context context, AttributeSet attrs) {
+    public FlingBehavior(final Context context, final AttributeSet attrs) {
         super(context, attrs);
     }
 
+    private boolean allowScroll = true;
+    private final Rect globalRect = new Rect();
+    private final List<Integer> skipInterceptionOfElements = Arrays.asList(
+            R.id.playQueuePanel, R.id.playbackSeekBar,
+            R.id.playPauseButton, R.id.playPreviousButton, R.id.playNextButton);
+
     @Override
-    public boolean onInterceptTouchEvent(CoordinatorLayout parent, AppBarLayout child, MotionEvent ev) {
+    public boolean onRequestChildRectangleOnScreen(
+            @NonNull final CoordinatorLayout coordinatorLayout, @NonNull final AppBarLayout child,
+            @NonNull final Rect rectangle, final boolean immediate) {
+        focusScrollRect.set(rectangle);
+
+        coordinatorLayout.offsetDescendantRectToMyCoords(child, focusScrollRect);
+
+        final int height = coordinatorLayout.getHeight();
+
+        if (focusScrollRect.top <= 0 && focusScrollRect.bottom >= height) {
+            // the child is too big to fit inside ourselves completely, ignore request
+            return false;
+        }
+
+        final int dy;
+
+        if (focusScrollRect.bottom > height) {
+            dy =  focusScrollRect.top;
+        } else if (focusScrollRect.top < 0) {
+            // scrolling up
+            dy = -(height - focusScrollRect.bottom);
+        } else {
+            // nothing to do
+            return false;
+        }
+
+        final int consumed = scroll(coordinatorLayout, child, dy, getMaxDragOffset(child), 0);
+
+        return consumed == dy;
+    }
+
+    public boolean onInterceptTouchEvent(final CoordinatorLayout parent, final AppBarLayout child,
+                                         final MotionEvent ev) {
+        for (final Integer element : skipInterceptionOfElements) {
+            final View view = child.findViewById(element);
+            if (view != null) {
+                final boolean visible = view.getGlobalVisibleRect(globalRect);
+                if (visible && globalRect.contains((int) ev.getRawX(), (int) ev.getRawY())) {
+                    allowScroll = false;
+                    return false;
+                }
+            }
+        }
+        allowScroll = true;
         switch (ev.getActionMasked()) {
             case MotionEvent.ACTION_DOWN:
                 // remove reference to old nested scrolling child
@@ -32,16 +88,37 @@ public final class FlingBehavior extends AppBarLayout.Behavior {
         return super.onInterceptTouchEvent(parent, child, ev);
     }
 
+    @Override
+    public boolean onStartNestedScroll(@NonNull final CoordinatorLayout parent,
+                                       @NonNull final AppBarLayout child,
+                                       @NonNull final View directTargetChild,
+                                       final View target,
+                                       final int nestedScrollAxes,
+                                       final int type) {
+        return allowScroll && super.onStartNestedScroll(
+                parent, child, directTargetChild, target, nestedScrollAxes, type);
+    }
+
+    @Override
+    public boolean onNestedFling(@NonNull final CoordinatorLayout coordinatorLayout,
+                                 @NonNull final AppBarLayout child,
+                                 @NonNull final View target, final float velocityX,
+                                 final float velocityY, final boolean consumed) {
+        return allowScroll && super.onNestedFling(
+                coordinatorLayout, child, target, velocityX, velocityY, consumed);
+    }
+
     @Nullable
     private OverScroller getScrollerField() {
         try {
-            Class<?> headerBehaviorType = this.getClass().getSuperclass().getSuperclass().getSuperclass();
+            final Class<?> headerBehaviorType = this.getClass()
+                    .getSuperclass().getSuperclass().getSuperclass();
             if (headerBehaviorType != null) {
-                Field field = headerBehaviorType.getDeclaredField("scroller");
+                final Field field = headerBehaviorType.getDeclaredField("scroller");
                 field.setAccessible(true);
                 return ((OverScroller) field.get(this));
             }
-        } catch (NoSuchFieldException | IllegalAccessException e) {
+        } catch (final NoSuchFieldException | IllegalAccessException e) {
             // ?
         }
         return null;
@@ -50,33 +127,37 @@ public final class FlingBehavior extends AppBarLayout.Behavior {
     @Nullable
     private Field getLastNestedScrollingChildRefField() {
         try {
-            Class<?> headerBehaviorType = this.getClass().getSuperclass().getSuperclass();
+            final Class<?> headerBehaviorType = this.getClass().getSuperclass().getSuperclass();
             if (headerBehaviorType != null) {
-                Field field = headerBehaviorType.getDeclaredField("lastNestedScrollingChildRef");
+                final Field field
+                        = headerBehaviorType.getDeclaredField("lastNestedScrollingChildRef");
                 field.setAccessible(true);
                 return field;
             }
-        } catch (NoSuchFieldException e) {
+        } catch (final NoSuchFieldException e) {
             // ?
         }
         return null;
     }
 
-    private void resetNestedScrollingChild(){
-        Field field = getLastNestedScrollingChildRefField();
-        if(field != null){
+    private void resetNestedScrollingChild() {
+        final Field field = getLastNestedScrollingChildRefField();
+        if (field != null) {
             try {
-                Object value = field.get(this);
-                if(value != null) field.set(this, null);
-            } catch (IllegalAccessException e) {
+                final Object value = field.get(this);
+                if (value != null) {
+                    field.set(this, null);
+                }
+            } catch (final IllegalAccessException e) {
                 // ?
             }
         }
     }
 
     private void stopAppBarLayoutFling() {
-        OverScroller scroller = getScrollerField();
-        if (scroller != null) scroller.forceFinished(true);
+        final OverScroller scroller = getScrollerField();
+        if (scroller != null) {
+            scroller.forceFinished(true);
+        }
     }
-
-}
+}

app/src/main/java/org/schabi/newpipe/ActivityCommunicator.java

@@ -23,17 +23,25 @@ package org.schabi.newpipe;
 /**
  * Singleton:
  * Used to send data between certain Activity/Services within the same process.
- * This can be considered as an ugly hack inside the Android universe. **/
+ * This can be considered as an ugly hack inside the Android universe.
+ **/
 public class ActivityCommunicator {
 
     private static ActivityCommunicator activityCommunicator;
+    private volatile Class returnActivity;
 
     public static ActivityCommunicator getCommunicator() {
-        if(activityCommunicator == null) {
+        if (activityCommunicator == null) {
             activityCommunicator = new ActivityCommunicator();
         }
         return activityCommunicator;
     }
 
-    public volatile Class returnActivity;
+    public Class getReturnActivity() {
+        return returnActivity;
+    }
+
+    public void setReturnActivity(final Class returnActivity) {
+        this.returnActivity = returnActivity;
+    }
 }

app/src/main/java/org/schabi/newpipe/BaseFragment.java

@@ -2,32 +2,31 @@ package org.schabi.newpipe;
 
 import android.content.Context;
 import android.os.Bundle;
-import androidx.annotation.NonNull;
-import androidx.fragment.app.Fragment;
-import androidx.fragment.app.FragmentManager;
-import androidx.appcompat.app.AppCompatActivity;
 import android.util.Log;
 import android.view.View;
 
+import androidx.annotation.NonNull;
+import androidx.appcompat.app.AppCompatActivity;
+import androidx.fragment.app.Fragment;
+import androidx.fragment.app.FragmentManager;
+
 import com.nostra13.universalimageloader.core.ImageLoader;
-import com.squareup.leakcanary.RefWatcher;
 
 import icepick.Icepick;
 import icepick.State;
+import leakcanary.AppWatcher;
 
 public abstract class BaseFragment extends Fragment {
+    public static final ImageLoader IMAGE_LOADER = ImageLoader.getInstance();
     protected final String TAG = getClass().getSimpleName() + "@" + Integer.toHexString(hashCode());
     protected final boolean DEBUG = MainActivity.DEBUG;
-
     protected AppCompatActivity activity;
-    public static final ImageLoader imageLoader = ImageLoader.getInstance();
-
-    //These values are used for controlling framgents when they are part of the frontpage
+    //These values are used for controlling fragments when they are part of the frontpage
     @State
     protected boolean useAsFrontPage = false;
-    protected boolean mIsVisibleToUser = false;
+    private boolean mIsVisibleToUser = false;
 
-    public void useAsFrontPage(boolean value) {
+    public void useAsFrontPage(final boolean value) {
         useAsFrontPage = value;
     }
 
@@ -36,7 +35,7 @@ public abstract class BaseFragment extends Fragment {
     //////////////////////////////////////////////////////////////////////////*/
 
     @Override
-    public void onAttach(Context context) {
+    public void onAttach(final Context context) {
         super.onAttach(context);
         activity = (AppCompatActivity) context;
     }
@@ -48,43 +47,49 @@ public abstract class BaseFragment extends Fragment {
     }
 
     @Override
-    public void onCreate(Bundle savedInstanceState) {
-        if (DEBUG) Log.d(TAG, "onCreate() called with: savedInstanceState = [" + savedInstanceState + "]");
+    public void onCreate(final Bundle savedInstanceState) {
+        if (DEBUG) {
+            Log.d(TAG, "onCreate() called with: "
+                    + "savedInstanceState = [" + savedInstanceState + "]");
+        }
         super.onCreate(savedInstanceState);
         Icepick.restoreInstanceState(this, savedInstanceState);
-        if (savedInstanceState != null) onRestoreInstanceState(savedInstanceState);
+        if (savedInstanceState != null) {
+            onRestoreInstanceState(savedInstanceState);
+        }
     }
 
 
     @Override
-    public void onViewCreated(View rootView, Bundle savedInstanceState) {
+    public void onViewCreated(final View rootView, final Bundle savedInstanceState) {
         super.onViewCreated(rootView, savedInstanceState);
         if (DEBUG) {
-            Log.d(TAG, "onViewCreated() called with: rootView = [" + rootView + "], savedInstanceState = [" + savedInstanceState + "]");
+            Log.d(TAG, "onViewCreated() called with: "
+                    + "rootView = [" + rootView + "], "
+                    + "savedInstanceState = [" + savedInstanceState + "]");
         }
         initViews(rootView, savedInstanceState);
         initListeners();
     }
 
     @Override
-    public void onSaveInstanceState(Bundle outState) {
+    public void onSaveInstanceState(final Bundle outState) {
         super.onSaveInstanceState(outState);
         Icepick.saveInstanceState(this, outState);
     }
 
-    protected void onRestoreInstanceState(@NonNull Bundle savedInstanceState) {
+    protected void onRestoreInstanceState(@NonNull final Bundle savedInstanceState) {
     }
 
     @Override
     public void onDestroy() {
         super.onDestroy();
 
-        RefWatcher refWatcher = App.getRefWatcher(getActivity());
-        if (refWatcher != null) refWatcher.watch(this);
+        AppWatcher.INSTANCE.getObjectWatcher().watch(this);
     }
 
     @Override
-    public void setUserVisibleHint(boolean isVisibleToUser) {
+    public void setUserVisibleHint(final boolean isVisibleToUser) {
         super.setUserVisibleHint(isVisibleToUser);
         mIsVisibleToUser = isVisibleToUser;
     }
@@ -93,7 +98,7 @@ public abstract class BaseFragment extends Fragment {
     // Init
     //////////////////////////////////////////////////////////////////////////*/
 
-    protected void initViews(View rootView, Bundle savedInstanceState) {
+    protected void initViews(final View rootView, final Bundle savedInstanceState) {
     }
 
     protected void initListeners() {
@@ -103,10 +108,12 @@ public abstract class BaseFragment extends Fragment {
     // Utils
     //////////////////////////////////////////////////////////////////////////*/
 
-    public void setTitle(String title) {
-        if (DEBUG) Log.d(TAG, "setTitle() called with: title = [" + title + "]");
-        if((!useAsFrontPage || mIsVisibleToUser)
-            && (activity != null && activity.getSupportActionBar() != null)) {
+    public void setTitle(final String title) {
+        if (DEBUG) {
+            Log.d(TAG, "setTitle() called with: title = [" + title + "]");
+        }
+        if ((!useAsFrontPage || mIsVisibleToUser)
+                && (activity != null && activity.getSupportActionBar() != null)) {
             activity.getSupportActionBar().setDisplayShowTitleEnabled(true);
             activity.getSupportActionBar().setTitle(title);
         }

app/src/main/java/org/schabi/newpipe/ExitActivity.java

@@ -1,4 +1,3 @@
-
 package org.schabi.newpipe;
 
 import android.annotation.SuppressLint;
@@ -27,22 +26,8 @@ import android.os.Bundle;
 
 public class ExitActivity extends Activity {
 
-    @SuppressLint("NewApi")
-    @Override
-    protected void onCreate(Bundle savedInstanceState) {
-        super.onCreate(savedInstanceState);
-
-        if (Build.VERSION.SDK_INT >= 21) {
-            finishAndRemoveTask();
-        } else {
-            finish();
-        }
-
-        System.exit(0);
-    }
-
-    public static void exitAndRemoveFromRecentApps(Activity activity) {
-        Intent intent = new Intent(activity, ExitActivity.class);
+    public static void exitAndRemoveFromRecentApps(final Activity activity) {
+        final Intent intent = new Intent(activity, ExitActivity.class);
 
         intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK
                 | Intent.FLAG_ACTIVITY_EXCLUDE_FROM_RECENTS
@@ -51,4 +36,18 @@ public class ExitActivity extends Activity {
 
         activity.startActivity(intent);
     }
+
+    @SuppressLint("NewApi")
+    @Override
+    protected void onCreate(final Bundle savedInstanceState) {
+        super.onCreate(savedInstanceState);
+
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
+            finishAndRemoveTask();
+        } else {
+            finish();
+        }
+
+        System.exit(0);
+    }
 }

app/src/main/java/org/schabi/newpipe/ImageDownloader.java

@@ -4,7 +4,7 @@ import android.annotation.SuppressLint;
 import android.content.Context;
 import android.content.SharedPreferences;
 import android.content.res.Resources;
-import android.preference.PreferenceManager;
+import androidx.preference.PreferenceManager;
 
 import com.nostra13.universalimageloader.core.download.BaseImageDownloader;
 
@@ -18,7 +18,7 @@ public class ImageDownloader extends BaseImageDownloader {
     private final SharedPreferences preferences;
     private final String downloadThumbnailKey;
 
-    public ImageDownloader(Context context) {
+    public ImageDownloader(final Context context) {
         super(context);
         this.resources = context.getResources();
         this.preferences = PreferenceManager.getDefaultSharedPreferences(context);
@@ -31,7 +31,7 @@ public class ImageDownloader extends BaseImageDownloader {
 
     @SuppressLint("ResourceType")
     @Override
-    public InputStream getStream(String imageUri, Object extra) throws IOException {
+    public InputStream getStream(final String imageUri, final Object extra) throws IOException {
         if (isDownloadingThumbnail()) {
             return super.getStream(imageUri, extra);
         } else {
@@ -39,7 +39,8 @@ public class ImageDownloader extends BaseImageDownloader {
         }
     }
 
-    protected InputStream getStreamFromNetwork(String imageUri, Object extra) throws IOException {
+    protected InputStream getStreamFromNetwork(final String imageUri, final Object extra)
+            throws IOException {
         final DownloaderImpl downloader = (DownloaderImpl) NewPipe.getDownloader();
         return downloader.stream(imageUri);
     }

app/src/main/java/org/schabi/newpipe/NewPipeDatabase.java

@@ -1,42 +1,54 @@
 package org.schabi.newpipe;
 
-import androidx.room.Room;
 import android.content.Context;
+import android.database.Cursor;
+
 import androidx.annotation.NonNull;
+import androidx.room.Room;
 
 import org.schabi.newpipe.database.AppDatabase;
 
 import static org.schabi.newpipe.database.AppDatabase.DATABASE_NAME;
-import static org.schabi.newpipe.database.Migrations.MIGRATION_11_12;
+import static org.schabi.newpipe.database.Migrations.MIGRATION_1_2;
+import static org.schabi.newpipe.database.Migrations.MIGRATION_2_3;
 
 public final class NewPipeDatabase {
-
     private static volatile AppDatabase databaseInstance;
 
     private NewPipeDatabase() {
         //no instance
     }
 
-    private static AppDatabase getDatabase(Context context) {
+    private static AppDatabase getDatabase(final Context context) {
         return Room
                 .databaseBuilder(context.getApplicationContext(), AppDatabase.class, DATABASE_NAME)
-                .addMigrations(MIGRATION_11_12)
-                .fallbackToDestructiveMigration()
+                .addMigrations(MIGRATION_1_2, MIGRATION_2_3)
                 .build();
     }
 
     @NonNull
-    public static AppDatabase getInstance(@NonNull Context context) {
+    public static AppDatabase getInstance(@NonNull final Context context) {
         AppDatabase result = databaseInstance;
         if (result == null) {
             synchronized (NewPipeDatabase.class) {
                 result = databaseInstance;
                 if (result == null) {
-                    databaseInstance = (result = getDatabase(context));
+                    databaseInstance = getDatabase(context);
+                    result = databaseInstance;
                 }
             }
         }
 
         return result;
     }
+
+    public static void checkpoint() {
+        if (databaseInstance == null) {
+            throw new IllegalStateException("database is not initialized");
+        }
+        final Cursor c = databaseInstance.query("pragma wal_checkpoint(full)", null);
+        if (c.moveToFirst() && c.getInt(0) == 1) {
+            throw new RuntimeException("Checkpoint was blocked from completing");
+        }
+    }
 }

app/src/main/java/org/schabi/newpipe/PanicResponderActivity.java

@@ -1,4 +1,3 @@
-
 package org.schabi.newpipe;
 
 import android.annotation.SuppressLint;
@@ -26,21 +25,22 @@ import android.os.Bundle;
  */
 
 public class PanicResponderActivity extends Activity {
-
     public static final String PANIC_TRIGGER_ACTION = "info.guardianproject.panic.action.TRIGGER";
 
     @SuppressLint("NewApi")
     @Override
-    protected void onCreate(Bundle savedInstanceState) {
+    protected void onCreate(final Bundle savedInstanceState) {
         super.onCreate(savedInstanceState);
-        Intent intent = getIntent();
+        final Intent intent = getIntent();
         if (intent != null && PANIC_TRIGGER_ACTION.equals(intent.getAction())) {
-            // TODO explicitly clear the search results once they are restored when the app restarts
-            // or if the app reloads the current video after being killed, that should be cleared also
+            // TODO: Explicitly clear the search results
+            //  once they are restored when the app restarts
+            //  or if the app reloads the current video after being killed,
+            //  that should be cleared also
             ExitActivity.exitAndRemoveFromRecentApps(this);
         }
 
-        if (Build.VERSION.SDK_INT >= 21) {
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
             finishAndRemoveTask();
         } else {
             finish();

app/src/main/java/org/schabi/newpipe/ReCaptchaActivity.java

@@ -1,24 +1,31 @@
 package org.schabi.newpipe;
 
 import android.content.Intent;
+import android.content.SharedPreferences;
 import android.os.Build;
 import android.os.Bundle;
-import androidx.core.app.NavUtils;
-import androidx.appcompat.app.ActionBar;
-import androidx.appcompat.app.AppCompatActivity;
-import androidx.appcompat.widget.Toolbar;
-
 import android.util.Log;
 import android.view.Menu;
 import android.view.MenuItem;
 import android.webkit.CookieManager;
+import android.webkit.WebResourceRequest;
 import android.webkit.WebSettings;
 import android.webkit.WebView;
 import android.webkit.WebViewClient;
 
+import androidx.annotation.NonNull;
+import androidx.annotation.Nullable;
+import androidx.annotation.RequiresApi;
+import androidx.appcompat.app.ActionBar;
+import androidx.appcompat.app.AppCompatActivity;
+import androidx.appcompat.widget.Toolbar;
+import androidx.core.app.NavUtils;
+import androidx.preference.PreferenceManager;
+
 import org.schabi.newpipe.util.ThemeHelper;
 
-import androidx.annotation.NonNull;
+import java.io.UnsupportedEncodingException;
+import java.net.URLDecoder;
 
 /*
  * Created by beneth <bmauduit@beneth.fr> on 06.12.16.
@@ -44,16 +51,17 @@ public class ReCaptchaActivity extends AppCompatActivity {
     public static final String RECAPTCHA_URL_EXTRA = "recaptcha_url_extra";
     public static final String TAG = ReCaptchaActivity.class.toString();
     public static final String YT_URL = "https://www.youtube.com";
+    public static final String RECAPTCHA_COOKIES_KEY = "recaptcha_cookies";
 
     private WebView webView;
     private String foundCookies = "";
 
     @Override
-    protected void onCreate(Bundle savedInstanceState) {
+    protected void onCreate(final Bundle savedInstanceState) {
         ThemeHelper.setTheme(this);
         super.onCreate(savedInstanceState);
         setContentView(R.layout.activity_recaptcha);
-        Toolbar toolbar = findViewById(R.id.toolbar);
+        final Toolbar toolbar = findViewById(R.id.toolbar);
         setSupportActionBar(toolbar);
 
         String url = getIntent().getStringExtra(RECAPTCHA_URL_EXTRA);
@@ -68,23 +76,47 @@ public class ReCaptchaActivity extends AppCompatActivity {
         webView = findViewById(R.id.reCaptchaWebView);
 
         // enable Javascript
-        WebSettings webSettings = webView.getSettings();
+        final WebSettings webSettings = webView.getSettings();
         webSettings.setJavaScriptEnabled(true);
 
         webView.setWebViewClient(new WebViewClient() {
+            @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
             @Override
-            public void onPageFinished(WebView view, String url) {
+            public boolean shouldOverrideUrlLoading(final WebView view,
+                                                    final WebResourceRequest request) {
+                final String url = request.getUrl().toString();
+                if (MainActivity.DEBUG) {
+                    Log.d(TAG, "shouldOverrideUrlLoading: request.url=" + url);
+                }
+
+                handleCookiesFromUrl(url);
+                return false;
+            }
+
+            @Override
+            public boolean shouldOverrideUrlLoading(final WebView view, final String url) {
+                if (MainActivity.DEBUG) {
+                    Log.d(TAG, "shouldOverrideUrlLoading: url=" + url);
+                }
+
+                handleCookiesFromUrl(url);
+                return false;
+            }
+
+            @Override
+            public void onPageFinished(final WebView view, final String url) {
                 super.onPageFinished(view, url);
-                handleCookies(url);
+                handleCookiesFromUrl(url);
             }
         });
 
         // cleaning cache, history and cookies from webView
         webView.clearCache(true);
         webView.clearHistory();
-        android.webkit.CookieManager cookieManager = CookieManager.getInstance();
+        final android.webkit.CookieManager cookieManager = CookieManager.getInstance();
         if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
-            cookieManager.removeAllCookies(aBoolean -> {});
+            cookieManager.removeAllCookies(aBoolean -> {
+            });
         } else {
             cookieManager.removeAllCookie();
         }
@@ -93,10 +125,10 @@ public class ReCaptchaActivity extends AppCompatActivity {
     }
 
     @Override
-    public boolean onCreateOptionsMenu(Menu menu) {
+    public boolean onCreateOptionsMenu(final Menu menu) {
         getMenuInflater().inflate(R.menu.menu_recaptcha, menu);
 
-        ActionBar actionBar = getSupportActionBar();
+        final ActionBar actionBar = getSupportActionBar();
         if (actionBar != null) {
             actionBar.setDisplayHomeAsUpEnabled(false);
             actionBar.setTitle(R.string.title_activity_recaptcha);
@@ -112,8 +144,8 @@ public class ReCaptchaActivity extends AppCompatActivity {
     }
 
     @Override
-    public boolean onOptionsItemSelected(MenuItem item) {
-        int id = item.getItemId();
+    public boolean onOptionsItemSelected(final MenuItem item) {
+        final int id = item.getItemId();
         switch (id) {
             case R.id.menu_item_done:
                 saveCookiesAndFinish();
@@ -124,37 +156,83 @@ public class ReCaptchaActivity extends AppCompatActivity {
     }
 
     private void saveCookiesAndFinish() {
-        handleCookies(webView.getUrl()); // try to get cookies of unclosed page
+        handleCookiesFromUrl(webView.getUrl()); // try to get cookies of unclosed page
+        if (MainActivity.DEBUG) {
+            Log.d(TAG, "saveCookiesAndFinish: foundCookies=" + foundCookies);
+        }
+
         if (!foundCookies.isEmpty()) {
+            // save cookies to preferences
+            final SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(
+                    getApplicationContext());
+            final String key = getApplicationContext().getString(R.string.recaptcha_cookies_key);
+            prefs.edit().putString(key, foundCookies).apply();
+
             // give cookies to Downloader class
-            DownloaderImpl.getInstance().setCookies(foundCookies);
+            DownloaderImpl.getInstance().setCookie(RECAPTCHA_COOKIES_KEY, foundCookies);
             setResult(RESULT_OK);
         }
 
-        Intent intent = new Intent(this, org.schabi.newpipe.MainActivity.class);
+        final Intent intent = new Intent(this, org.schabi.newpipe.MainActivity.class);
         intent.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP);
         NavUtils.navigateUpTo(this, intent);
     }
 
 
+    private void handleCookiesFromUrl(@Nullable final String url) {
+        if (MainActivity.DEBUG) {
+            Log.d(TAG, "handleCookiesFromUrl: url=" + (url == null ? "null" : url));
+        }
 
-    private void handleCookies(String url) {
-        String cookies = CookieManager.getInstance().getCookie(url);
-        if (MainActivity.DEBUG) Log.d(TAG, "handleCookies: url=" + url + "; cookies=" + (cookies == null ? "null" : cookies));
-        if (cookies == null) return;
+        if (url == null) {
+            return;
+        }
 
-        addYoutubeCookies(cookies);
-        // add other methods to extract cookies here
+        final String cookies = CookieManager.getInstance().getCookie(url);
+        handleCookies(cookies);
+
+        // sometimes cookies are inside the url
+        final int abuseStart = url.indexOf("google_abuse=");
+        if (abuseStart != -1) {
+            final int abuseEnd = url.indexOf("+path");
+
+            try {
+                String abuseCookie = url.substring(abuseStart + 13, abuseEnd);
+                abuseCookie = URLDecoder.decode(abuseCookie, "UTF-8");
+                handleCookies(abuseCookie);
+            } catch (UnsupportedEncodingException | StringIndexOutOfBoundsException e) {
+                if (MainActivity.DEBUG) {
+                    e.printStackTrace();
+                    Log.d(TAG, "handleCookiesFromUrl: invalid google abuse starting at "
+                            + abuseStart + " and ending at " + abuseEnd + " for url " + url);
+                }
+            }
+        }
     }
 
-    private void addYoutubeCookies(@NonNull String cookies) {
-        if (cookies.contains("s_gl=") || cookies.contains("goojf=") || cookies.contains("VISITOR_INFO1_LIVE=")) {
+    private void handleCookies(@Nullable final String cookies) {
+        if (MainActivity.DEBUG) {
+            Log.d(TAG, "handleCookies: cookies=" + (cookies == null ? "null" : cookies));
+        }
+
+        if (cookies == null) {
+            return;
+        }
+
+        addYoutubeCookies(cookies);
+        // add here methods to extract cookies for other services
+    }
+
+    private void addYoutubeCookies(@NonNull final String cookies) {
+        if (cookies.contains("s_gl=") || cookies.contains("goojf=")
+                || cookies.contains("VISITOR_INFO1_LIVE=")
+                || cookies.contains("GOOGLE_ABUSE_EXEMPTION=")) {
             // youtube seems to also need the other cookies:
             addCookie(cookies);
         }
     }
 
-    private void addCookie(String cookie) {
+    private void addCookie(final String cookie) {
         if (foundCookies.contains(cookie)) {
             return;
         }

app/src/main/java/org/schabi/newpipe/about/AboutActivity.java

@@ -1,51 +1,67 @@
 package org.schabi.newpipe.about;
 
 import android.content.Context;
-import android.content.Intent;
-import android.net.Uri;
 import android.os.Bundle;
-import com.google.android.material.tabs.TabLayout;
+import android.view.LayoutInflater;
+import android.view.MenuItem;
+import android.view.View;
+import android.view.ViewGroup;
+import android.widget.TextView;
+
+import androidx.appcompat.app.AppCompatActivity;
+import androidx.appcompat.widget.Toolbar;
 import androidx.fragment.app.Fragment;
 import androidx.fragment.app.FragmentManager;
 import androidx.fragment.app.FragmentPagerAdapter;
 import androidx.fragment.app.FragmentStatePagerAdapter;
 import androidx.viewpager.widget.PagerAdapter;
 import androidx.viewpager.widget.ViewPager;
-import androidx.appcompat.app.AppCompatActivity;
-import androidx.appcompat.widget.Toolbar;
-import android.view.LayoutInflater;
-import android.view.Menu;
-import android.view.MenuItem;
-import android.view.View;
-import android.view.ViewGroup;
-import android.widget.TextView;
+
+import com.google.android.material.tabs.TabLayout;
 
 import org.schabi.newpipe.BuildConfig;
 import org.schabi.newpipe.R;
 import org.schabi.newpipe.util.ThemeHelper;
 
 import static org.schabi.newpipe.util.Localization.assureCorrectAppLanguage;
+import static org.schabi.newpipe.util.ShareUtils.openUrlInBrowser;
 
 public class AboutActivity extends AppCompatActivity {
-
     /**
-     * List of all software components
+     * List of all software components.
      */
     private static final SoftwareComponent[] SOFTWARE_COMPONENTS = new SoftwareComponent[]{
-            new SoftwareComponent("Giga Get", "2014 - 2015", "Peter Cai", "https://github.com/PaperAirplane-Dev-Team/GigaGet", StandardLicenses.GPL2),
-            new SoftwareComponent("NewPipe Extractor", "2017 - 2020", "Christian Schabesberger", "https://github.com/TeamNewPipe/NewPipeExtractor", StandardLicenses.GPL3),
-            new SoftwareComponent("Jsoup", "2017", "Jonathan Hedley", "https://github.com/jhy/jsoup", StandardLicenses.MIT),
-            new SoftwareComponent("Rhino", "2015", "Mozilla", "https://www.mozilla.org/rhino/", StandardLicenses.MPL2),
-            new SoftwareComponent("ACRA", "2013", "Kevin Gaudin", "http://www.acra.ch", StandardLicenses.APACHE2),
-            new SoftwareComponent("Universal Image Loader", "2011 - 2015", "Sergey Tarasevich", "https://github.com/nostra13/Android-Universal-Image-Loader", StandardLicenses.APACHE2),
-            new SoftwareComponent("CircleImageView", "2014 - 2020", "Henning Dodenhof", "https://github.com/hdodenhof/CircleImageView", StandardLicenses.APACHE2),
-            new SoftwareComponent("NoNonsense-FilePicker", "2016", "Jonas Kalderstam", "https://github.com/spacecowboy/NoNonsense-FilePicker", StandardLicenses.MPL2),
-            new SoftwareComponent("ExoPlayer", "2014 - 2020", "Google Inc", "https://github.com/google/ExoPlayer", StandardLicenses.APACHE2),
-            new SoftwareComponent("RxAndroid", "2015 - 2018", "The RxAndroid authors", "https://github.com/ReactiveX/RxAndroid", StandardLicenses.APACHE2),
-            new SoftwareComponent("RxJava", "2016 - 2020", "RxJava Contributors", "https://github.com/ReactiveX/RxJava", StandardLicenses.APACHE2),
-            new SoftwareComponent("RxBinding", "2015 - 2018", "Jake Wharton", "https://github.com/JakeWharton/RxBinding", StandardLicenses.APACHE2),
-            new SoftwareComponent("PrettyTime", "2012 - 2020", "Lincoln Baxter, III", "https://github.com/ocpsoft/prettytime", StandardLicenses.APACHE2),
-            new SoftwareComponent("Markwon", "2017 - 2020", "Noties", "https://github.com/noties/Markwon", StandardLicenses.APACHE2)
+            new SoftwareComponent("Giga Get", "2014 - 2015", "Peter Cai",
+                    "https://github.com/PaperAirplane-Dev-Team/GigaGet", StandardLicenses.GPL2),
+            new SoftwareComponent("NewPipe Extractor", "2017 - 2020", "Christian Schabesberger",
+                    "https://github.com/TeamNewPipe/NewPipeExtractor", StandardLicenses.GPL3),
+            new SoftwareComponent("Jsoup", "2017", "Jonathan Hedley",
+                    "https://github.com/jhy/jsoup", StandardLicenses.MIT),
+            new SoftwareComponent("Rhino", "2015", "Mozilla",
+                    "https://www.mozilla.org/rhino/", StandardLicenses.MPL2),
+            new SoftwareComponent("ACRA", "2013", "Kevin Gaudin",
+                    "http://www.acra.ch", StandardLicenses.APACHE2),
+            new SoftwareComponent("Universal Image Loader", "2011 - 2015", "Sergey Tarasevich",
+                    "https://github.com/nostra13/Android-Universal-Image-Loader",
+                    StandardLicenses.APACHE2),
+            new SoftwareComponent("CircleImageView", "2014 - 2020", "Henning Dodenhof",
+                    "https://github.com/hdodenhof/CircleImageView", StandardLicenses.APACHE2),
+            new SoftwareComponent("NoNonsense-FilePicker", "2016", "Jonas Kalderstam",
+                    "https://github.com/spacecowboy/NoNonsense-FilePicker", StandardLicenses.MPL2),
+            new SoftwareComponent("ExoPlayer", "2014 - 2020", "Google Inc",
+                    "https://github.com/google/ExoPlayer", StandardLicenses.APACHE2),
+            new SoftwareComponent("RxAndroid", "2015 - 2018", "The RxAndroid authors",
+                    "https://github.com/ReactiveX/RxAndroid", StandardLicenses.APACHE2),
+            new SoftwareComponent("RxJava", "2016 - 2020", "RxJava Contributors",
+                    "https://github.com/ReactiveX/RxJava", StandardLicenses.APACHE2),
+            new SoftwareComponent("RxBinding", "2015 - 2018", "Jake Wharton",
+                    "https://github.com/JakeWharton/RxBinding", StandardLicenses.APACHE2),
+            new SoftwareComponent("PrettyTime", "2012 - 2020", "Lincoln Baxter, III",
+                    "https://github.com/ocpsoft/prettytime", StandardLicenses.APACHE2),
+            new SoftwareComponent("Markwon", "2017 - 2020", "Noties",
+                    "https://github.com/noties/Markwon", StandardLicenses.APACHE2),
+            new SoftwareComponent("Groupie", "2016", "Lisa Wray",
+                    "https://github.com/lisawray/groupie", StandardLicenses.MIT)
     };
 
     /**
@@ -64,7 +80,7 @@ public class AboutActivity extends AppCompatActivity {
     private ViewPager mViewPager;
 
     @Override
-    protected void onCreate(Bundle savedInstanceState) {
+    protected void onCreate(final Bundle savedInstanceState) {
         assureCorrectAppLanguage(this);
         super.onCreate(savedInstanceState);
         ThemeHelper.setTheme(this);
@@ -72,7 +88,7 @@ public class AboutActivity extends AppCompatActivity {
 
         setContentView(R.layout.activity_about);
 
-        Toolbar toolbar = findViewById(R.id.toolbar);
+        final Toolbar toolbar = findViewById(R.id.toolbar);
         setSupportActionBar(toolbar);
         getSupportActionBar().setDisplayHomeAsUpEnabled(true);
         // Create the adapter that will return a fragment for each of the three
@@ -83,15 +99,13 @@ public class AboutActivity extends AppCompatActivity {
         mViewPager = findViewById(R.id.container);
         mViewPager.setAdapter(mSectionsPagerAdapter);
 
-        TabLayout tabLayout = findViewById(R.id.tabs);
+        final TabLayout tabLayout = findViewById(R.id.tabs);
         tabLayout.setupWithViewPager(mViewPager);
     }
 
-
     @Override
-    public boolean onOptionsItemSelected(MenuItem item) {
-
-        int id = item.getItemId();
+    public boolean onOptionsItemSelected(final MenuItem item) {
+        final int id = item.getItemId();
 
         switch (id) {
             case android.R.id.home:
@@ -106,62 +120,58 @@ public class AboutActivity extends AppCompatActivity {
      * A placeholder fragment containing a simple view.
      */
     public static class AboutFragment extends Fragment {
-
-        public AboutFragment() {
-        }
+        public AboutFragment() { }
 
         /**
-         * Returns a new instance of this fragment for the given section
-         * number.
+         * Created a new instance of this fragment for the given section number.
+         *
+         * @return New instance of {@link AboutFragment}
          */
         public static AboutFragment newInstance() {
             return new AboutFragment();
         }
 
         @Override
-        public View onCreateView(LayoutInflater inflater, ViewGroup container,
-                                 Bundle savedInstanceState) {
-            View rootView = inflater.inflate(R.layout.fragment_about, container, false);
-            Context context = this.getContext();
+        public View onCreateView(final LayoutInflater inflater, final ViewGroup container,
+                                 final Bundle savedInstanceState) {
+            final View rootView = inflater.inflate(R.layout.fragment_about, container, false);
+            final Context context = this.getContext();
 
-            TextView version = rootView.findViewById(R.id.app_version);
+            final TextView version = rootView.findViewById(R.id.app_version);
             version.setText(BuildConfig.VERSION_NAME);
 
-            View githubLink = rootView.findViewById(R.id.github_link);
-            githubLink.setOnClickListener(nv -> openWebsite(context.getString(R.string.github_url), context));
+            final View githubLink = rootView.findViewById(R.id.github_link);
+            githubLink.setOnClickListener(nv ->
+                    openUrlInBrowser(context, context.getString(R.string.github_url)));
 
-            View donationLink = rootView.findViewById(R.id.donation_link);
-            donationLink.setOnClickListener(v -> openWebsite(context.getString(R.string.donation_url), context));
+            final View donationLink = rootView.findViewById(R.id.donation_link);
+            donationLink.setOnClickListener(v ->
+                    openUrlInBrowser(context, context.getString(R.string.donation_url)));
 
-            View websiteLink = rootView.findViewById(R.id.website_link);
-            websiteLink.setOnClickListener(nv -> openWebsite(context.getString(R.string.website_url), context));
+            final View websiteLink = rootView.findViewById(R.id.website_link);
+            websiteLink.setOnClickListener(nv ->
+                    openUrlInBrowser(context, context.getString(R.string.website_url)));
 
-            View privacyPolicyLink = rootView.findViewById(R.id.privacy_policy_link);
-            privacyPolicyLink.setOnClickListener(v -> openWebsite(context.getString(R.string.privacy_policy_url), context));
+            final View privacyPolicyLink = rootView.findViewById(R.id.privacy_policy_link);
+            privacyPolicyLink.setOnClickListener(v ->
+                    openUrlInBrowser(context, context.getString(R.string.privacy_policy_url)));
 
             return rootView;
         }
 
-        private void openWebsite(String url, Context context) {
-            Intent intent = new Intent(Intent.ACTION_VIEW, Uri.parse(url));
-            context.startActivity(intent);
-        }
-
     }
 
-
     /**
      * A {@link FragmentPagerAdapter} that returns a fragment corresponding to
      * one of the sections/tabs/pages.
      */
     public class SectionsPagerAdapter extends FragmentPagerAdapter {
-
-        public SectionsPagerAdapter(FragmentManager fm) {
-            super(fm);
+        public SectionsPagerAdapter(final FragmentManager fm) {
+            super(fm, BEHAVIOR_RESUME_ONLY_CURRENT_FRAGMENT);
         }
 
         @Override
-        public Fragment getItem(int position) {
+        public Fragment getItem(final int position) {
             switch (position) {
                 case 0:
                     return AboutFragment.newInstance();
@@ -178,7 +188,7 @@ public class AboutActivity extends AppCompatActivity {
         }
 
         @Override
-        public CharSequence getPageTitle(int position) {
+        public CharSequence getPageTitle(final int position) {
             switch (position) {
                 case 0:
                     return getString(R.string.tab_about);