Bump urllib3 from 1.25.2 to 1.25.8 (#162)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.25.2 to 1.25.8.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.25.2...1.25.8)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

.gitignore

@@ -1,3 +1,7 @@
 .envrc
 __pycache__
+*.pyc
 venv
+deletefb.log
+test.sh
+chromedriver

.travis.yml

@@ -0,0 +1,17 @@
+sudo: required
+services:
+  - docker
+env:
+  global:
+  #   setup these vars under settings at https://travis-ci.com
+  #   - REGISTRY_USER=${REGISTRY_USER}
+  #   - REGISTRY_PASS=${REGISTRY_PASS}
+  #   - IMAGE_NAME=${REGISTRY_USER}/deletefb
+    
+script:
+  - echo "${REGISTRY_PASS}" | docker login -u "${REGISTRY_USER}" --password-stdin
+  - docker build -t "${REGISTRY_USER}/${IMAGE_NAME}" .
+  - docker images
+  - docker push ${REGISTRY_USER}/${IMAGE_NAME}
+on: 
+  branch: master

CONTRIBUTING.md

@@ -0,0 +1,13 @@
+### How to contribute
+
+## Dependencies
+If you are adding any new dependencies, please make sure that both `requirements.txt` and `setup.py` have been updated. Please read [this](https://caremad.io/posts/2013/07/setup-vs-requirement/) if you are confused about the difference between `requirements.txt` and the `install_requires` section.
+
+## Virtualenv
+Always develop with virtualenv, as well as test with `pip install --user .`. This helps make sure implicit dependencies aren't accidentally introduced, and makes sure the average user will be more likely to run it without issues.
+
+## Pull requests
+Feel free to make a pull request! Make sure to give a brief overview of what you did, and why you think it is useful. If you are fixing a specific bug or resolving an issue, then make sure to reference it in your PR.
+
+## Coding style
+Try to be consistent with the existing codebase as much as possible. Things should be modularized. Don't repeat yourself if possible, but don't add needless complexity. Straightforward is often better than clever and optimized.

Dockerfile

@@ -0,0 +1,58 @@
+# To run, just type "make", or 
+
+# docker build -t deletefb . 
+# docker run -ti --rm \
+#     -e DISPLAY=$DISPLAY \
+#     -v /tmp/.X11-unix:/tmp/.X11-unix \
+#     --cap-add=SYS_ADMIN \
+#     --cap-add=NET_ADMIN \
+#     --cpuset-cpus 0 \
+#     --memory 4GB \
+#     -v /tmp/.X11-unix:/tmp/.X11-unix \
+#     -e DISPLAY=unix:0 \
+#     --device /dev/snd \
+#     --device /dev/dri \
+#     -v /dev/shm:/dev/shm  \
+#     deletefb  -e mail="your@email.com" -e pass="Y0Ur*P4ss" -e url="http://facebook.com/your-username" deletefb:latest
+
+FROM debian:stable-slim
+    
+    RUN apt-get update &&  \
+     apt-get install -y \
+     git \
+     python3 \
+     python3-pip \
+     libcanberra-gtk-module \
+     curl \ 
+     sudo \ 
+     vim  \
+     unzip \
+     chromium \
+     chromium-driver
+
+#creating new user
+    ENV user deletefb
+    RUN export uid=1000 gid=1000 && \
+    mkdir -p /home/${user} && \
+    echo "${user}:x:${uid}:${gid}:${user},,,:/home/${user}:/bin/bash" >> /etc/passwd && \
+    echo "${user}:x:${uid}:" >> /etc/group && \
+    echo "${user} ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/${user} && \
+    chmod 0440 /etc/sudoers.d/${user} && \
+    chown ${uid}:${gid} -R /home/${user} && \
+    usermod -aG sudo ${user}
+
+
+# deletefb install
+    USER ${user}
+    WORKDIR /home/${user}
+    
+    ARG email  
+    ARG pass  
+    ARG url  
+    #ARG --conversations
+
+    RUN pip3 install --user delete-facebook-posts
+    RUN pip3 install --user selenium attrs pybloom_live
+
+    ADD run.sh /tmp/run.sh
+    ENTRYPOINT [ "/tmp/run.sh" ]

FUNDING.yml

@@ -0,0 +1,2 @@
+github: [weskerfoot]
+custom: "39qHYvjVcMCNFr3RPAVetci9mKjzYGTQPz"

Makefile

@@ -0,0 +1,24 @@
+# Makefile
+
+NAME:= deletefb
+
+.PHONY: all build run
+
+all:  build run
+
+build:
+	@docker build -t $(NAME) .   
+
+run:
+	@read -p "Enter your Facebook email: " email && read -p "Enter your Facebook password: " password && read -p "Enter your Facebook username: " username  && docker run -ti --rm \
+    -e DISPLAY=$$DISPLAY \
+    -v /tmp/.X11-unix:/tmp/.X11-unix \
+    --cap-add=SYS_ADMIN \
+    --cap-add=NET_ADMIN \
+    --cpuset-cpus 0 \
+    --device /dev/dri \
+    -v /dev/shm:/dev/shm  \
+    -e EMAIL="$$email" \
+    -e PASS="$$password" \
+    -e URL="https://facebook.com/$$username" \
+    $(NAME):latest

README.md

@@ -1,3 +1,7 @@
+WARNING:
+This currently only works for English language Facebook accounts, due to the lack of a usable API.
+Also, year by year deletion is currently broken. Feel free to fork or make pull requests.
+
 ## Why?
 
 I needed a simple and reliable way to delete Facebook posts. There are
@@ -17,26 +21,55 @@ online presence and not have to worry about what you wrote from years ago.
 Personally, I did this so I would feel less attached to my Facebook profile
 (and hence feel the need to use it less).
 
+## Dependencies
+- This tool requires at least Python 3.7 in order to run.
+- A recent copy of Chrome or Chromium installed and available in your `$PATH`
+
 ## Installation
 You have several options to run it.
-1) Install from PyPI with `pip3 install --user delete-facebook-posts`
+1) Install from PyPI with `pip3 install --user delete-facebook-posts` (recommended you do this in a virtualenv to avoid incompatibilities)
 2) Clone this repo and run `pip3 install --user .` or do `pip3 install --user
 git+https://github.com/weskerfoot/DeleteFB.git`
 3) Set up a Python virtualenv, activate it, and run `pip3 install -r requirements.txt`, then you can just run `python -m deletefb.deletefb` in the DeleteFB directory.
+4) Use the docker image (experimental) by running `make` after checking this repository out with git. There is also an image built and published automatically at `wjak56/deletefb:latest`
+
+## Chromedriver
+The tool will attempt to detect the version of Chrome that you have installed and download the appropriate chromedriver. It is possible that it might fail to find your chrome version if you are running on Windows. If that is the case, please try running the docker version.
 
 ## How To Use It
+```
+usage: deletefb [-h] [-M {wall,unlike_pages,conversations}] -E EMAIL [-P PASSWORD] -U PROFILE_URL [-F TWO_FACTOR_TOKEN] [-H] [--no-archive] [-Y YEAR]
+                [-B CHROMEBIN]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -M {wall,unlike_pages,conversations}, --mode {wall,unlike_pages,conversations}
+                        The mode you want to run in. Default is `wall' which deletes wall posts
+  -E EMAIL, --email EMAIL
+                        Your email address associated with the account
+  -P PASSWORD, --password PASSWORD
+                        Your Facebook password
+  -U PROFILE_URL, --profile-url PROFILE_URL
+                        The link to your Facebook profile, e.g. https://www.facebook.com/your.name
+  -F TWO_FACTOR_TOKEN, --two-factor TWO_FACTOR_TOKEN
+                        The code generated by your 2FA device for Facebook
+  -H, --headless        Run browser in headless mode (no gui)
+  --no-archive          Turn off archiving (on by default)
+  -Y YEAR, --year YEAR  The year(s) you want posts deleted.
+  -B CHROMEBIN, --chromebin CHROMEBIN
+                        Optional path to the Google Chrome (or Chromium) binary
+```
 
 * Make sure that you have a recent version of Python 3.x installed (preferably
-  3.6 or greater)
+  3.7 or greater)
 * Make sure that you have Google Chrome installed and that it is up to date
-* Also install the chromedriver for Selenium. See [here](https://sites.google.com/a/chromium.org/chromedriver/home) for an explanation of what the chromedriver does.
+* The tool will attempt to automatically install chromedriver for Selenium. See [here](https://sites.google.com/a/chromium.org/chromedriver/home) for an explanation of what the chromedriver does. You may have to manually install it if auto-install fails.
   * On Linux, it will be called something like `chromium-chromedriver` or just
     `chromium`.
   * On MacOS, it will be available via brew, with the following commands:
 
     ```
-    brew tap homebrew/cask;
-    brew cask install chromedriver
+    brew install chromedriver
     ```
 
 * Run `deletefb -E 'youremail@example.org' -P 'yourfacebookpassword' -U 'https://www.facebook.com/your.profile.url'`
@@ -47,12 +80,7 @@ git+https://github.com/weskerfoot/DeleteFB.git`
   everything. You may safely minimize the chrome window without breaking it.
 
 ## Login
-* The tool will log in using the credentials passed to it. It will wait until
-  the page "https://www.facebook.com/" is loaded in order to avoid any issues
-  with logging in. If you pass a 2FA token explicitly with the `-F` option,
-  then it will try to enter that for you. If there are any issues, it simply
-  pauses indefinitely to allow the user to resolve the problems, and then
-  continues execution.
+* The tool will log in using the credentials passed to it. It will wait until the page `https://www.facebook.com/` is loaded in order to avoid any issues with logging in. If you pass a 2FA token explicitly with the `-F` option, then it will try to enter that for you. If there are any issues, it simply pauses indefinitely to allow the user to resolve the problems, and then continues execution.
 
 ## 2FA
 * It is recommended that you disable Two-Factor Authentication temporarily
@@ -67,14 +95,7 @@ git+https://github.com/weskerfoot/DeleteFB.git`
 * You may also pass in a code by using the `-F` argument, e.g. `-F 111111`.
 
 ## Delete By Year
-* The tool supports passing the `--year` flag in order to delete wall posts by
-  year. E.g. `-Y 2010` would delete posts from the year 2010. It is incompatible with any mode other than `wall`.
-
-## Unlike Pages
-* You may use `-M unlike_pages` to unlike all of your pages. The names of the
-  pages will be archived (unless archival is turned off), and this option
-  conflicts with the year option. This will only unlike your *pages* that you
-  have liked. It will *not* unlike anything else (like books or movies).
+* The tool supports passing the `--year` flag in order to delete/archive by year. E.g. `-Y 2010` would only affect posts from 2010.
 
 ## Archival
 * The tool will archive everything being deleted by default in `.log` files.

deletefb/deletefb.py

@@ -4,6 +4,9 @@ from .tools.config import settings
 from .tools.likes import unlike_pages
 from .tools.login import login
 from .tools.wall import delete_posts
+from .tools.conversations import traverse_conversations
+from .tools.comments import delete_comments
+from .quit_driver import quit_driver_and_reap_children
 
 import argparse
 import getpass
@@ -21,7 +24,7 @@ def run_delete():
         default="wall",
         dest="mode",
         type=str,
-        choices=["wall", "unlike_pages"],
+        choices=["wall", "unlike_pages", "conversations"],
         help="The mode you want to run in. Default is `wall' which deletes wall posts"
     )
 
@@ -87,12 +90,22 @@ def run_delete():
         help="The year(s) you want posts deleted."
     )
 
+    parser.add_argument(
+        "-B",
+        "--chromebin",
+        required=False,
+        default=False,
+        dest="chromebin",
+        type=str,
+        help="Optional path to the Google Chrome (or Chromium) binary"
+    )
+
     args = parser.parse_args()
 
     settings["ARCHIVE"] = not args.archive_off
 
-    if args.year and args.mode != "wall":
-        parser.error("The --year option is only supported in wall mode")
+    if args.year and args.mode not in ("wall", "conversations"):
+        parser.error("The --year option is not supported in this mode")
 
     args_user_password = args.password or getpass.getpass('Enter your password: ')
 
@@ -100,21 +113,31 @@ def run_delete():
         user_email_address=args.email,
         user_password=args_user_password,
         is_headless=args.is_headless,
-        two_factor_token=args.two_factor_token
+        two_factor_token=args.two_factor_token,
+        chrome_binary_path=args.chromebin
     )
 
-    if args.mode == "wall":
-        delete_posts(
-            driver,
-            args.profile_url,
-            year=args.year
-        )
-
-    elif args.mode == "unlike_pages":
-        unlike_pages(driver, args.profile_url)
-    else:
-        print("Please enter a valid mode")
-        sys.exit(1)
+    try:
+        if args.mode == "wall":
+            delete_posts(
+                driver,
+                args.profile_url,
+                year=args.year
+            )
+
+        elif args.mode == "unlike_pages":
+            unlike_pages(driver, args.profile_url)
+
+        elif args.mode == "conversations":
+            traverse_conversations(driver, year=args.year)
+
+        else:
+            print("Please enter a valid mode")
+            sys.exit(1)
+    except BaseException as e:
+        print(e)
+        if driver:
+            quit_driver_and_reap_children(driver)
 
 if __name__ == "__main__":
     run_delete()

deletefb/exceptions.py

@@ -0,0 +1,5 @@
+class UnknownOSException(Exception):
+    pass
+
+class ChromeError(Exception):
+    pass

deletefb/quit_driver.py

@@ -0,0 +1,13 @@
+import os
+
+def quit_driver_and_reap_children(driver):
+    """
+    Reaps child processes by waiting until they exit.
+    """
+    driver.quit()
+    try:
+        pid = True
+        while pid:
+            pid = os.waitpid(-1, os.WNOHANG)
+    except ChildProcessError:
+        pass

deletefb/tools/archive.py

@@ -1,13 +1,23 @@
 from .config import settings
 from contextlib import contextmanager
 from pathlib import Path
+from datetime import datetime
+from time import time
 
 import attr
+import cattr
 import json
+import typing
+
+TIME_FORMAT = "%Y-%m-%d %H:%M:%S"
 
 # Used to avoid duplicates in the log
 from pybloom_live import BloomFilter
 
+cattr.register_unstructure_hook(
+    datetime, lambda dt: datetime.strftime(dt, format=TIME_FORMAT)
+)
+
 def make_filter():
     return BloomFilter(
         capacity=settings["MAX_POSTS"],
@@ -27,28 +37,44 @@ class Archive:
         """
         Archive an object
         """
-        print("Archiving {0}".format(content))
+
+        if hasattr(content, 'name'):
+            print("Archiving {0}".format(content.name))
 
         if content.name not in self._bloom_filter:
-            self.archive_file.write(json.dumps(attr.asdict(content)) + "\n")
+            self.archive_file.write(json.dumps(cattr.unstructure(content),
+                                               indent=4,
+                                               sort_keys=True) + "\n")
+
             self._bloom_filter.add(content.name)
         return
 
+
+class FakeArchive:
+    def archive(self, content):
+        """
+        Do not archive an object
+        """
+        return
+
+
 @contextmanager
 def archiver(archive_type):
+    if not settings["ARCHIVE"]:
+        yield FakeArchive()
+    else:
+        archive_file = open(
+            str((Path(".") / Path(archive_type).name).with_suffix(".log.{0}".format(time()))),
+            mode="ta",
+            buffering=1
+        )
 
-    archive_file = open(
-        (Path(".") / Path(archive_type).name).with_suffix(".log"),
-        mode="ta",
-        buffering=1
-    )
-
-    archiver_instance = Archive(
-        archive_type=archive_type,
-        archive_file=archive_file
-    )
+        archiver_instance = Archive(
+            archive_type=archive_type,
+            archive_file=archive_file
+        )
 
-    try:
-        yield archiver_instance
-    finally:
-        archive_file.close()
+        try:
+            yield archiver_instance
+        finally:
+            archive_file.close()

deletefb/tools/chrome_driver.py

@@ -0,0 +1,167 @@
+from ..exceptions import UnknownOSException, ChromeError
+from .common import NO_CHROME_DRIVER
+from clint.textui import puts, colored
+from selenium import webdriver
+from selenium.common.exceptions import WebDriverException
+from shutil import which
+from subprocess import check_output
+from urllib.request import urlretrieve
+from appdirs import AppDirs
+from ..version import version
+from os.path import exists
+
+import os, sys, stat, platform
+import progressbar
+import re
+import zipfile
+import requests
+import pathlib
+
+cache_dir = AppDirs("DeleteFB", version=version).user_cache_dir
+
+try:
+    pathlib.Path(cache_dir).mkdir(parents=True, exist_ok=True)
+except FileExistsError:
+    pass
+
+def extract_zip(filename, chrome_maj_version):
+    """
+    Uses zipfile package to extract a single zipfile
+    :param filename:
+    :return: new filename
+    """
+
+    # Remove any leftover unversioned chromedriver
+    try:
+        os.remove(f"{cache_dir}/chromedriver")
+    except FileNotFoundError:
+        pass
+
+    try:
+        _file = zipfile.ZipFile(filename, 'r')
+    except FileNotFoundError:
+        puts(colored.red(f"{filename} Does not exist"))
+        sys.exit(1)
+
+    # Save the name of the new file
+    new_file_name = f"{cache_dir}/{_file.namelist()[0] + chrome_maj_version}"
+
+    # Extract the file and make it executable
+    _file.extractall(path=cache_dir)
+
+    # Rename the filename to a versioned one
+    os.rename(f"{cache_dir}/chromedriver", f"{cache_dir}/chromedriver{chrome_maj_version}")
+
+    driver_stat = os.stat(new_file_name)
+    os.chmod(new_file_name, driver_stat.st_mode | stat.S_IEXEC)
+
+    _file.close()
+    os.remove(filename)
+    return new_file_name
+
+
+def setup_selenium(options, chrome_binary_path):
+    try:
+        # try letting Selenium find the driver (in PATH)
+        return webdriver.Chrome(options=options)
+    except WebDriverException:
+        # Configures selenium to use a custom path
+        driver_path = get_webdriver(chrome_binary_path)
+        return webdriver.Chrome(executable_path=driver_path, options=options)
+
+def parse_version(output):
+    """
+    Attempt to extract version number from chrome version string.
+    """
+    return [c for c in re.split('([0-9]+)\.?', output.decode("utf-8")) if all(d.isdigit() for d in c) and c][0]
+
+def get_chrome_version(chrome_binary_path=None):
+    """
+    Extract the chrome major version.
+    """
+    driver_locations = [which(loc) for loc in ["google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome.exe"]]
+
+    for location in driver_locations:
+        if location:
+            return parse_version(check_output([location, "--version"]).strip())
+    return None
+
+def construct_driver_url(chrome_binary_path=None):
+    """
+    Construct a URL to download the Chrome Driver.
+    """
+
+    platform_string = platform.system()
+    chrome_drivers = {
+        "Windows" : "https://chromedriver.storage.googleapis.com/{0}/chromedriver_win32.zip",
+        "Darwin" : "https://chromedriver.storage.googleapis.com/{0}/chromedriver_mac64.zip",
+        "Linux" : "https://chromedriver.storage.googleapis.com/{0}/chromedriver_linux64.zip"
+    }
+
+    version = get_chrome_version()
+
+    if version is None:
+        raise ChromeError("Chrome version not found")
+
+    latest_release_url = "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_{0}".format(version)
+
+    return version, chrome_drivers.get(platform_string).format(requests.get(latest_release_url).text)
+
+    # First, construct a LATEST_RELEASE URL using Chrome's major version number.
+    # For example, with Chrome version 73.0.3683.86, use URL "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_73".
+    # Try to download a small file from this URL. If it successful, the file contains the ChromeDriver version to use.
+    # If the above step failed, reduce the Chrome major version by 1 and try again.
+    # For example, with Chrome version 75.0.3745.4, use URL "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_74"
+    #  to download a small file, which contains the ChromeDriver version to use.
+    # You can also use ChromeDriver Canary build.
+
+def get_webdriver(chrome_binary_path):
+    """
+     Ensure a webdriver is available
+     If Not, Download it.
+    """
+
+    # Download it according to the current machine
+    chrome_maj_version, chrome_webdriver = construct_driver_url(chrome_binary_path)
+
+    driver_path = f"{cache_dir}/chromedriver{chrome_maj_version}"
+
+    if exists(driver_path):
+        return driver_path
+
+    if not chrome_webdriver:
+        raise UnknownOSException("Unknown Operating system platform")
+
+    global total_size
+
+    def show_progress(*res):
+        global total_size
+        pbar = None
+        downloaded = 0
+        block_num, block_size, total_size = res
+
+        if not pbar:
+            pbar = progressbar.ProgressBar(maxval=total_size)
+            pbar.start()
+        downloaded += block_num * block_size
+
+        if downloaded < total_size:
+            pbar.update(downloaded)
+        else:
+            pbar.finish()
+
+    puts(colored.yellow("Downloading Chrome Webdriver"))
+    file_name = f"{cache_dir}/{chrome_webdriver.split('/')[-1]}"
+    response = urlretrieve(chrome_webdriver, file_name, show_progress)
+
+    if int(response[1].get("Content-Length")) == total_size:
+        puts(colored.green("Completed downloading the Chrome Driver."))
+
+        return extract_zip(file_name, chrome_maj_version)
+
+    else:
+        puts(colored.red("An error Occurred While trying to download the driver."))
+        # remove the downloaded file and exit
+        os.remove(file_name)
+        sys.stderr.write(NO_CHROME_DRIVER)
+        sys.exit(1)

deletefb/tools/comments.py

@@ -0,0 +1,17 @@
+from .archive import archiver
+from ..types import Comment
+from .common import SELENIUM_EXCEPTIONS, logger, click_button
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+from selenium.webdriver.support.ui import WebDriverWait
+
+LOG = logger(__name__)
+
+def delete_comments(driver, profile_url):
+    """
+    Remove all comments on posts
+    """
+
+    driver.get("{0}/allactivity?privacy_source=activity_log&category_key=commentscluster".format(profile_url))
+
+    wait = WebDriverWait(driver, 20)

deletefb/tools/common.py

@@ -1,4 +1,7 @@
 from os.path import isfile
+from selenium.webdriver.support.ui import WebDriverWait
+from selenium.webdriver.support import expected_conditions as EC
+from selenium.webdriver.common.by import By
 from selenium.common.exceptions import (
     NoSuchElementException,
     StaleElementReferenceException,
@@ -9,6 +12,7 @@ import json
 import logging
 import logging.config
 import os
+import urllib.parse as urlparse
 
 SELENIUM_EXCEPTIONS = (
     NoSuchElementException,
@@ -19,13 +23,27 @@ SELENIUM_EXCEPTIONS = (
 def click_button(driver, el):
     """
     Click a button using Javascript
-    Args:
-        driver: seleniumrequests.Chrome Driver instance
-    Returns:
-        None
     """
     driver.execute_script("arguments[0].click();", el)
 
+def scroll_until_element_exists(driver, xpath_expr):
+    while True:
+        driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
+        try:
+            element = driver.find_element_by_xpath(xpath_expr)
+        except SELENIUM_EXCEPTIONS:
+            continue
+        break
+
+def scroll_to(driver, el):
+    """
+    Scroll an element into view, using JS
+    """
+    try:
+        driver.execute_script("arguments[0].scrollIntoView();", el)
+    except SELENIUM_EXCEPTIONS:
+        return
+
 def logger(name):
     """
     Args:
@@ -45,7 +63,31 @@ def logger(name):
         logging.config.dictConfig(config["logging"])
     return logging.getLogger(name)
 
+def wait_xpath(driver, expr, timeout=20):
+    """
+    Takes an XPath expression, and waits at most `timeout` seconds until it exists
+    """
+    wait = WebDriverWait(driver, timeout)
+    try:
+        wait.until(EC.presence_of_element_located((By.XPATH, expr)))
+    except SELENIUM_EXCEPTIONS:
+        return
+
+def force_mobile(url):
+    """
+    Force a url to use the mobile site.
+    """
+    parsed = urlparse.urlparse(url)
+    # Ensure a protocol is given (needed by selenium).
+    scheme = parsed.scheme or "https"
+    return urlparse.urlunparse((scheme,
+                                "mobile.facebook.com",
+                                parsed.path,
+                                parsed.params,
+                                parsed.query,
+                                parsed.fragment))
+
 NO_CHROME_DRIVER = """
-You need to install the chromedriver for Selenium\n
+You need to manually install the chromedriver for Selenium\n
 Please see this link https://github.com/weskerfoot/DeleteFB#how-to-use-it\n
 """

deletefb/tools/conversations.py

@@ -0,0 +1,184 @@
+from .archive import archiver
+from ..types import Conversation, Message
+from .common import SELENIUM_EXCEPTIONS, logger, click_button, wait_xpath
+from .config import settings
+from selenium.webdriver.common.action_chains import ActionChains
+from selenium.webdriver.support.ui import Select
+from pendulum import now
+from json import loads
+from time import sleep
+
+import lxml.html as lxh
+
+LOG = logger(__name__)
+
+def get_conversations(driver):
+    """
+    Get a list of conversations
+    """
+
+    wait_xpath(driver, "//div[@id=\"threadlist_rows\"]")
+
+    # This function *cannot* be a generator
+    # Otherwise elements will become stale
+    conversations = []
+
+    while True:
+        for convo in driver.find_elements_by_xpath("//a"):
+            url = convo.get_attribute("href")
+
+            date = None
+
+            if url and "messages/read" in url:
+                try:
+                    date = convo.find_element_by_xpath("../../..//abbr").text
+                    conversation_name = convo.find_element_by_xpath("../../../div/div/header/h3").text.strip()
+                    assert(conversation_name)
+                    assert(url)
+                except (SELENIUM_EXCEPTIONS + (AssertionError,)):
+                    continue
+
+                conversations.append(
+                    Conversation(
+                        url=url,
+                        date=date,
+                        name=conversation_name
+                    )
+                )
+
+        try:
+            next_url = (driver.find_element_by_id("see_older_threads").
+                        find_element_by_xpath("a").
+                        get_attribute("href"))
+
+            print("next_url", next_url)
+
+        except SELENIUM_EXCEPTIONS as e:
+            print(e)
+            break
+        if not next_url:
+            break
+        driver.get(next_url)
+
+    return conversations
+
+def parse_conversation(driver):
+    """
+    Extracts all messages in a conversation
+    """
+
+    for msg in lxh.fromstring(driver.page_source).xpath("//div[@class='msg']/div"):
+        data_store = loads(msg.get("data-store"))
+        msg_text = msg.text_content()
+
+        yield Message(
+                name=data_store.get("author"),
+                content=msg_text,
+                date=data_store.get("timestamp")
+              )
+
+def get_images(driver):
+    """
+    Gets all links to images in a messenger conversation
+    Removes duplicates
+    """
+    for img in set(lxh.fromstring(driver.page_source).xpath("//img")):
+        yield img.get("src")
+
+def get_convo(driver, convo):
+    """
+    Get all of the messages/images for a given conversation
+    Returns a list of messages and a list of image links
+    """
+    driver.get(convo.url)
+
+    wait_xpath(driver, "//*[contains(text(), 'See Older Messages')]")
+
+    # Expand conversation until we've reached the beginning
+    while True:
+        try:
+            see_older = driver.find_element_by_xpath("//*[contains(text(), 'See Older Messages')]")
+        except SELENIUM_EXCEPTIONS:
+            break
+
+        if not see_older:
+            break
+
+        try:
+            click_button(driver, see_older)
+        except SELENIUM_EXCEPTIONS:
+            continue
+
+    messages = list(parse_conversation(driver))
+    image_links = list(set(get_images(driver)))
+    return (messages, image_links)
+
+def delete_conversation(driver, convo):
+    """
+    Deletes a conversation
+    """
+
+    actions = ActionChains(driver)
+
+    menu_select = Select(driver.find_element_by_xpath("//select/option[contains(text(), 'Delete')]/.."))
+
+    for i, option in enumerate(menu_select.options):
+        if option.text.strip() == "Delete":
+            menu_select.select_by_index(i)
+            break
+
+    wait_xpath(driver, "//h2[contains(text(), 'Delete conversation')]")
+    delete_button = driver.find_element_by_xpath("//a[contains(text(), 'Delete')][@role='button']")
+    actions.move_to_element(delete_button).click().perform()
+
+    return
+
+def extract_convo(driver, convo):
+    """
+    Extract messages and image links from a conversation
+    Return a new Conversation instance
+    """
+    result = get_convo(driver, convo)
+
+    if not result:
+        return None
+
+    messages, image_links = result
+
+    convo.messages = messages
+    convo.image_links = image_links
+
+    return convo
+
+def traverse_conversations(driver, year=None):
+    """
+    Remove all conversations within a specified range
+    """
+
+    driver.get("https://mobile.facebook.com/messages/?pageNum=1&selectable&see_older_newer=1")
+
+    convos = get_conversations(driver)
+
+    with archiver("conversations") as archive_convo:
+        for convo in convos:
+            # If the year is set and there is a date
+            # Then we want to only look at convos from this year
+
+            if year and convo.date:
+                if convo.date.year == int(year):
+                    extract_convo(driver, convo)
+
+                    if settings["ARCHIVE"]:
+                        archive_convo.archive(convo)
+
+                    delete_conversation(driver, convo)
+
+            # Otherwise we're looking at all convos
+            elif not year:
+                extract_convo(driver, convo)
+
+                if settings["ARCHIVE"]:
+                    archive_convo.archive(convo)
+
+                delete_conversation(driver, convo)
+

deletefb/tools/likes.py

@@ -1,6 +1,6 @@
 from .archive import archiver
 from ..types import Page
-from .common import SELENIUM_EXCEPTIONS, logger, click_button
+from .common import SELENIUM_EXCEPTIONS, logger, click_button, wait_xpath, force_mobile, scroll_to, scroll_until_element_exists
 from selenium.webdriver.common.by import By
 from selenium.webdriver.support import expected_conditions as EC
 from selenium.webdriver.support.ui import WebDriverWait
@@ -18,17 +18,25 @@ def load_likes(driver, profile_url):
         None
     """
 
-    driver.get("{0}/likes_all".format(profile_url))
+    likes_link_xpath = "//div[normalize-space(text())='Likes']/../..//a[contains(@href, 'app_section')]"
 
-    wait = WebDriverWait(driver, 20)
+    all_likes_link_xpath = "//div[normalize-space(text())='All Likes']/../../../..//a[contains(@href, 'app_collection')]"
 
-    try:
-        wait.until(
-            EC.presence_of_element_located((By.CSS_SELECTOR, ".PageLikeButton"))
-        )
-    except SELENIUM_EXCEPTIONS:
-        LOG.exception("Traceback of load_likes")
-        return
+    driver.get(force_mobile("{0}/about".format(profile_url)))
+
+    scroll_until_element_exists(driver, "//div[text()='Likes']")
+
+    likes_link_el = driver.find_element_by_xpath(likes_link_xpath)
+
+    driver.get(likes_link_el.get_attribute("href"))
+
+    wait_xpath(driver, all_likes_link_xpath)
+
+    all_likes_link_el = driver.find_element_by_xpath(all_likes_link_xpath)
+
+    all_likes_link = all_likes_link_el.get_attribute("href")
+
+    driver.get(all_likes_link)
 
 def get_page_links(driver):
     """
@@ -39,8 +47,7 @@ def get_page_links(driver):
     Returns:
         List of URLs to pages
     """
-    pages = driver.find_elements_by_xpath("//li//div/div/a[contains(@class, 'lfloat')]")
-
+    pages = driver.find_elements_by_xpath("//header/..//a")
     return [page.get_attribute("href").replace("www", "mobile") for page in pages]
 
 def unlike_page(driver, url, archive=None):
@@ -60,7 +67,7 @@ def unlike_page(driver, url, archive=None):
 
     print("Unliking {0}".format(url))
 
-    wait = WebDriverWait(driver, 20)
+    wait = WebDriverWait(driver, 5)
 
     try:
         wait.until(
@@ -70,19 +77,14 @@ def unlike_page(driver, url, archive=None):
         # Something went wrong with this page, so skip it
         return
 
-    button = driver.find_element_by_xpath("//*[text()='Liked']")
-
-    # Click the "Liked" button to open up "Unlike"
-    click_button(driver, button)
+    driver.find_element_by_xpath("//*[text()='Liked']/../../../..").click()
 
     wait.until(
         EC.presence_of_element_located((By.XPATH, "//*[text()='Unlike']"))
     )
 
     # There should be an "Unlike" button now, click it
-    unlike_button = driver.find_element_by_xpath("//*[text()='Unlike']/..")
-
-    click_button(driver, unlike_button)
+    driver.find_element_by_xpath("//*[text()='Unlike']/..").click()
 
     if archive:
         archive(Page(name=url))

deletefb/tools/login.py

@@ -1,15 +1,15 @@
-from .common import NO_CHROME_DRIVER
+from .chrome_driver import setup_selenium
 from selenium.common.exceptions import NoSuchElementException
 from selenium.webdriver.chrome.options import Options
-from seleniumrequests import Chrome
+from ..quit_driver import quit_driver_and_reap_children
 
-import sys
 import time
 
 def login(user_email_address,
           user_password,
           is_headless,
-          two_factor_token):
+          two_factor_token,
+          chrome_binary_path=None):
     """
     Attempts to log into Facebook
     Returns a driver object
@@ -27,80 +27,78 @@ def login(user_email_address,
     chrome_options = Options()
     prefs = {"profile.default_content_setting_values.notifications": 2, 'disk-cache-size': 4096}
     chrome_options.add_experimental_option("prefs", prefs)
+
+    if chrome_binary_path:
+        chrome_options.binary_location = chrome_binary_path
+
     chrome_options.add_argument("start-maximized")
 
     if is_headless:
         chrome_options.add_argument('--headless')
         chrome_options.add_argument('--disable-gpu')
+        chrome_options.add_argument('--disabled-features=VizDisplayCompositor')
+        chrome_options.add_argument('--dump-dom')
+        chrome_options.add_argument('--no-sandbox')
         chrome_options.add_argument('log-level=2')
 
+    driver = setup_selenium(chrome_options, chrome_binary_path)
     try:
-        driver = Chrome(options=chrome_options)
-    except Exception as e:
-        # The user does not have chromedriver installed
-        # Tell them to install it
-        sys.stderr.write(str(e))
-        sys.stderr.write(NO_CHROME_DRIVER)
-        sys.exit(1)
-
-    driver.implicitly_wait(10)
-
-    driver.get("https://www.facebook.com/login/device-based/regular/login/?login_attempt=1&lwv=110")
-
-    email = "email"
-    password = "pass"
-    login_button = "loginbutton"
-    approvals_code = "approvals_code"
-
-    emailelement = driver.find_element_by_name(email)
-    passwordelement = driver.find_element_by_name(password)
-
-    emailelement.send_keys(user_email_address)
-    passwordelement.send_keys(user_password)
-
-    loginelement = driver.find_element_by_id(login_button)
-    loginelement.click()
-
-    # Defaults to no 2fa
-    has_2fa = False
-
-    try:
-        # If this element exists, we've reached a 2FA page
-        driver.find_element_by_xpath("//form[@class=\"checkpoint\"]")
-        driver.find_element_by_xpath("//input[@name=\"approvals_code\"]")
-        has_2fa = True
-    except NoSuchElementException:
-        has_2fa = "two-factor authentication" in driver.page_source.lower() or has_2fa
-
-    if has_2fa:
-        print("""
-            Two-Factor Auth is enabled.
-            Please file an issue at https://github.com/weskerfoot/DeleteFB/issues if you run into any problems
-        """)
-
-    if two_factor_token and has_2fa:
-        twofactorelement = driver.find_element_by_name(approvals_code)
-        twofactorelement.send_keys(two_factor_token)
-
-        # Submits after the code is passed into the form, does not validate 2FA code.
-        contelement = driver.find_element_by_id("checkpointSubmitButton")
-        contelement.click()
-
-        # Defaults to saving this new browser, this occurs on each new automated login.
-        save_browser = driver.find_element_by_id("checkpointSubmitButton")
-        save_browser.click()
-    elif has_2fa:
-        # Allow time to enter 2FA code
-        print("Pausing to enter 2FA code")
-        time.sleep(35)
-        print("Continuing execution")
-    else:
-        pass
-
-    # block until we have reached the main page
-    # print a message warning the user
-    while driver.current_url != "https://www.facebook.com/":
-        print("Execution blocked: Please navigate to https://www.facebook.com to continue")
-        time.sleep(5)
-
-    return driver
+        driver.implicitly_wait(10)
+
+        driver.get("https://www.facebook.com/login/device-based/regular/login/?login_attempt=1&lwv=110")
+
+        email = "email"
+        password = "pass"
+        login_button = "loginbutton"
+        approvals_code = "approvals_code"
+
+        driver.find_element_by_name(email).send_keys(user_email_address)
+        driver.find_element_by_name(password).send_keys(user_password)
+        driver.find_element_by_id(login_button).click()
+
+        # Defaults to no 2fa
+        has_2fa = False
+
+        try:
+            # If this element exists, we've reached a 2FA page
+            driver.find_element_by_xpath("//form[@class=\"checkpoint\"]")
+            driver.find_element_by_xpath("//input[@name=\"approvals_code\"]")
+            has_2fa = True
+        except NoSuchElementException:
+            has_2fa = "two-factor authentication" in driver.page_source.lower() or has_2fa
+
+        if has_2fa:
+            print("""
+                Two-Factor Auth is enabled.
+                Please file an issue at https://github.com/weskerfoot/DeleteFB/issues if you run into any problems
+            """)
+
+        if two_factor_token and has_2fa:
+            twofactorelement = driver.find_element_by_name(approvals_code)
+            twofactorelement.send_keys(two_factor_token)
+
+            # Submits after the code is passed into the form, does not validate 2FA code.
+            contelement = driver.find_element_by_id("checkpointSubmitButton")
+            contelement.click()
+
+            # Defaults to saving this new browser, this occurs on each new automated login.
+            save_browser = driver.find_element_by_id("checkpointSubmitButton")
+            save_browser.click()
+        elif has_2fa:
+            # Allow time to enter 2FA code
+            print("Pausing to enter 2FA code")
+            time.sleep(35)
+            print("Continuing execution")
+        else:
+            pass
+
+        # block until we have reached the main page
+        # print a message warning the user
+        while driver.current_url != "https://www.facebook.com/":
+            print("Execution blocked: Please navigate to https://www.facebook.com to continue")
+            time.sleep(5)
+
+        return driver
+    except BaseException as e:
+        print('An exception occurred: {}'.format(e))
+        quit_driver_and_reap_children(driver)

deletefb/tools/wall.py

@@ -1,6 +1,6 @@
 from ..types import Post
 from .archive import archiver
-from .common import SELENIUM_EXCEPTIONS, click_button
+from .common import SELENIUM_EXCEPTIONS, click_button, wait_xpath, force_mobile
 from .config import settings
 from selenium.webdriver.common.action_chains import ActionChains
 
@@ -24,24 +24,43 @@ def delete_posts(driver,
     if year is not None:
         user_profile_url = "{0}/timeline?year={1}".format(user_profile_url, year)
 
+    user_profile_url = force_mobile(user_profile_url)
+
     driver.get(user_profile_url)
 
-    for _ in range(MAX_POSTS):
-        post_button_sel = "_4xev"
+    finished = False
+
+    with archiver("wall") as archive_wall_post:
+        for _ in range(MAX_POSTS):
+            if finished:
+                break
+            post_button_sel = "_4s19"
+
+            post_content_sel = "userContent"
+            post_timestamp_sel = "timestampContent"
 
-        post_content_sel = "userContent"
-        post_timestamp_sel = "timestampContent"
+            confirmation_button_exp = "//div[contains(@data-sigil, 'undo-content')]//*/a[contains(@href, 'direct_action_execute')]"
 
-        button_types = ["FeedDeleteOption", "HIDE_FROM_TIMELINE", "UNTAG"]
+            # Cannot return a text node, so it returns the parent.
+            # Tries to be pretty resilient against DOM re-organizations
+            timestamp_exp = "//article//*/header//*/div/a[contains(@href, 'story_fbid')]//text()/.."
+
+            button_types = ["Delete post", "Remove tag", "Hide from timeline", "Hide from profile"]
 
-        with archiver("wall") as archive_wall_post:
             while True:
                 try:
-                    timeline_element = driver.find_element_by_class_name(post_button_sel)
+                    try:
+                        timeline_element = driver.find_element_by_xpath("//div[@data-sigil='story-popup-causal-init']/a")
+                    except SELENIUM_EXCEPTIONS:
+                        print("Could not find any posts")
+                        finished = True
+                        break
 
-                    post_content_element = driver.find_element_by_class_name(post_content_sel)
-                    post_content_ts = driver.find_element_by_class_name(post_timestamp_sel)
+                    post_content_element = driver.find_element_by_xpath("//article/div[@class='story_body_container']/div")
+                    post_content_ts = driver.find_element_by_xpath(timestamp_exp)
 
+                    if not (post_content_element or post_content_ts):
+                        break
 
                     # Archive the post
                     archive_wall_post.archive(
@@ -54,28 +73,36 @@ def delete_posts(driver,
                     actions = ActionChains(driver)
                     actions.move_to_element(timeline_element).click().perform()
 
-                    menu = driver.find_element_by_css_selector("#globalContainer > div.uiContextualLayerPositioner.uiLayer > div")
-                    actions.move_to_element(menu).perform()
+                    # Wait until the buttons show up
+                    wait_xpath(driver, "//*[contains(@data-sigil, 'removeStoryButton')]")
 
                     delete_button = None
 
+                    # Search for visible buttons in priority order
+                    # Delete -> Untag -> Hide
                     for button_type in button_types:
                         try:
-                            delete_button = menu.find_element_by_xpath("//a[@data-feed-option-name=\"{0}\"]".format(button_type))
+                            delete_button = driver.find_element_by_xpath("//*[text()='{0}']".format(button_type))
+                            if not delete_button.is_displayed():
+                                continue
                             break
-                        except SELENIUM_EXCEPTIONS:
+                        except SELENIUM_EXCEPTIONS as e:
+                            print(e)
                             continue
 
                     if not delete_button:
                         print("Could not find anything to delete")
                         break
 
-                    actions.move_to_element(delete_button).click().perform()
-                    confirmation_button = driver.find_element_by_class_name("layerConfirm")
+                    click_button(driver, delete_button)
+                    wait_xpath(driver, confirmation_button_exp)
+                    confirmation_button = driver.find_element_by_xpath(confirmation_button_exp)
 
+                    print(confirmation_button)
                     click_button(driver, confirmation_button)
 
-                except SELENIUM_EXCEPTIONS:
+                except SELENIUM_EXCEPTIONS as e:
+                    print(e)
                     continue
                 else:
                     break

deletefb/types.py

@@ -1,29 +1,55 @@
 import attr
 import uuid
-import datetime
+import pendulum
 
-def timestamp_now():
+from datetime import datetime
+
+def convert_date(text):
     """
-    Returns: a timestamp for this instant, in ISO 8601 format
+    Tries to parse a date into a DateTime instance
+    Returns `None` if it cannot be parsed
     """
-    return datetime.datetime.isoformat(datetime.datetime.now())
+    try:
+        return pendulum.from_format(text, "DD/M/YYYY")
+    except ValueError:
+        try:
+            return (pendulum.from_format(text, "DD MMM")
+                    .set(year=pendulum.now().year))
+        except ValueError:
+            return None
 
 # Data type definitions of posts and comments
 @attr.s
 class Post:
     content = attr.ib()
-    comments = attr.ib(default=[])
-    date = attr.ib(factory=timestamp_now)
+    comments = attr.ib(factory=list)
+    date = attr.ib(factory=pendulum.now)
     name = attr.ib(factory=lambda: uuid.uuid4().hex)
 
 @attr.s
 class Comment:
     commenter = attr.ib()
     content = attr.ib()
-    date = attr.ib(factory=timestamp_now)
+    date = attr.ib(factory=pendulum.now)
     name = attr.ib(factory=lambda: uuid.uuid4().hex)
 
+@attr.s
+class Conversation:
+    url = attr.ib()
+    name = attr.ib()
+    date : datetime = attr.ib(converter=convert_date)
+    messages = attr.ib(factory=list)
+    image_links = attr.ib(factory=list)
+
+@attr.s
+class Message:
+    name = attr.ib()
+    content = attr.ib()
+
+    # Remove the last 3 digits from FB's dates. They are not standard.
+    date : datetime = attr.ib(converter=lambda t: pendulum.from_timestamp(int(str(t)[0:-3])))
+
 @attr.s
 class Page:
     name = attr.ib()
-    date = attr.ib(factory=timestamp_now)
+    date = attr.ib(factory=pendulum.now)

deletefb/version.py

@@ -0,0 +1,6 @@
+import pkg_resources  # part of setuptools
+
+try:
+    version = pkg_resources.require("delete-facebook-posts")[0].version
+except pkg_resources.DistributionNotFound:
+    version = "source"

requirements.txt

@@ -1,13 +1,32 @@
-attrs==19.1.0
+appdirs==1.4.3
+args==0.1.0
+attrs==20.3.0
 bitarray==0.9.3
+bleach==3.3.0
+cattrs==1.1.2
 certifi==2018.11.29
 chardet==3.0.4
+clint==0.5.1
+docutils==0.14
 idna==2.8
+lxml==4.6.3
+pendulum==2.0.5
+pkginfo==1.5.0.1
+progressbar==2.5
 pybloom-live==3.0.0
+Pygments==2.7.4
+python-dateutil==2.8.0
+pytzdata==2019.2
+readme-renderer==24.0
 requests==2.22.0
 requests-file==1.4.3
+requests-toolbelt==0.9.1
 selenium==3.141.0
 selenium-requests==1.3
 six==1.12.0
 tldextract==2.2.0
-urllib3==1.25.2
+tqdm==4.32.2
+twine==1.13.0
+typing==3.7.4
+urllib3==1.25.8
+webencodings==0.5.1

run.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+/usr/bin/python3 -m deletefb.deletefb -E $EMAIL -P $PASS -U $URL

setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
 
 setuptools.setup(
     name="delete-facebook-posts",
-    version="1.1.7",
+    version="1.1.17",
     author="Wesley Kerfoot",
     author_email="wes@wesk.tech",
     description="A Selenium Script to Delete Facebook Posts",
@@ -24,7 +24,13 @@ setuptools.setup(
         "selenium-requests",
         "requests",
         "pybloom-live",
-        "attrs"
+        "attrs>=20.3.0",
+        "cattrs>=1.1.2",
+        "lxml",
+        "pendulum",
+        "clint",
+        "progressbar",
+        "appdirs"
     ],
     classifiers= [
         "Programming Language :: Python :: 3",