Merge PR #757 (Improve loadable module handling)

NEWS.md

@@ -4,8 +4,27 @@ Atheme Services 7.3 Development Notes
 There have been various changes since the last non-point release, most of which
 are not documented here yet.
 
+GUARANTEED COMPATIBILITY BREAKAGE
+---------------------------------
+
+- The `loadmodule` lines in the configuration file no longer take the
+  "modules/" prefix. The example configuration file at
+  `dist/atheme.conf.example` has been updated accordingly.
+
+  So, if you currently have: `loadmodule "modules/nickserv/cert";`
+  Then you will need to change this to: `loadmodule "nickserv/cert";`
+
+  This can be done en masse with a `sed(1)` invocation on your configuration
+  file. Please take a backup first:
+
+  ```
+  $ cp atheme.conf atheme.conf.bak
+  $ sed -r -i -e 's|^loadmodule "modules/|loadmodule "|g' atheme.conf
+  ```
+
 POTENTIAL COMPATIBILITY BREAKAGE
 --------------------------------
+
 - Services now accepts nicknames up to 50 characters in length, because some
   IRCds like Charybdis do (if so configured). However, if you actually *use*
   nicknames on your network greater than *31* characters, your database WILL
@@ -16,12 +35,12 @@ POTENTIAL COMPATIBILITY BREAKAGE
   Atheme <= 7.2, this module has been replaced with 4 other modules (2 of which
   provide compatibility for the removed module). The module you need to load
   depends upon the operating system Atheme was being used on; if it was Mac OS
-  then you need to load `modules/crypto/crypt3-des` instead. If it was any
-  other operating system, then you need to load `modules/crypto/crypt3-md5`
-  instead. Note that these 2 modules are compatibility modules; they can only
-  verify existing encrypted passwords, they cannot encrypt new ones. You must
-  load an encryption-capable crypto module. Please see the Password Hashing
-  Modules section of `dist/atheme.conf.example`.
+  then you need to load `crypto/crypt3-des` instead. If it was any other
+  operating system, then you need to load `crypto/crypt3-md5` instead. Note
+  that these 2 modules are compatibility modules; they can only verify existing
+  encrypted passwords, they cannot encrypt new ones. You must load an
+  encryption-capable crypto module. Please see the Password Hashing Modules
+  section of `dist/atheme.conf.example`.
 
 - If you (still) use legacy password crypto (verify-only) modules (`anope-*`,
   `base64`, `crypt3-des`, `crypt3-md5`, `ircservices`, `raw*`), then you MUST
@@ -29,27 +48,34 @@ POTENTIAL COMPATIBILITY BREAKAGE
   will NOT be compiled or installed. The presence of this flag can be confirmed
   at the bottom of the `configure` output; "Legacy Crypto Modules".
 
-- The `modules/nickserv/cracklib` module has been renamed to
-  `modules/nickserv/pwquality` because it is now capable of using `libpasswdqc`
-  as well. The corresponding configuration item `nickserv::cracklib_warn` has
-  been renamed to `nickserv::pwquality_warn_only` too.
+- The `nickserv/cracklib` module has been renamed to `nickserv/pwquality`
+  because it is now capable of using `libpasswdqc` as well. The corresponding
+  configuration item `nickserv::cracklib_warn` has been renamed to
+  `nickserv::pwquality_warn_only` too.
 
-- The `modules/gameserv/happyfarm` module has been removed, as it was never
-  completely finished and never worked anyway. Please remove this module from
-  your configuration file, regardless of the version of services you are using.
+- The `gameserv/happyfarm` module has been removed, as it was never completely
+  finished and never worked anyway. Please remove this module from your
+  configuration file, regardless of the version of services you are using.
 
-- The `modules/operserv/override` module has been removed. It did not provide
+- The `operserv/override` module has been removed. It did not provide
   sufficient transparency to users while providing a great potential for abuse.
   Additionally, it caused crashes if used with certain commands. Any legitimate
   use of this module should be possible to replace with a more specific command
-  (such as `modules/chanserv/fflags`). If you encounter a use case that cannot
-  be replaced, please report a bug to let us know.
+  (such as `chanserv/fflags`). If you encounter a use case that cannot be
+  replaced, please report a bug to let us know.
 
-- The `modules/operserv/set` module has been broken up into individual modules.
+- The `operserv/set` module has been broken up into individual modules.
   Existing loadmodule configurations will continue to work, but you will
   receive a module deprecation warning if you load it. Please see the
   `dist/atheme.conf.example` file for the new submodule names.
 
+- The `operserv/modinspect`, `operserv/modload`, `operserv/modreload` and
+  `operserv/modunload` modules have been removed, and replaced with a single
+  module `operserv/modmanager`. Please migrate your configuration if you were
+  previously using any of these 4 modules, and rely upon the `operserv::access`
+  configuration block to restrict usage of commands if you were e.g. previously
+  running without `operserv/modload` loaded.
+
 - The NickServ DROP command no longer requires the user's account password as
   an argument.
 
@@ -160,8 +186,8 @@ ChanServ
 
 NickServ
 --------
-- Port `contrib/ns_waitreg` to `modules/nickserv/waitreg`
-- Port `contrib/ns_listlogins` to `modules/nickserv/listlogins`
+- Port `contrib/ns_waitreg` to `nickserv/waitreg`
+- Port `contrib/ns_listlogins` to `nickserv/listlogins`
 - Blame a specific channel when a NickServ `REGAIN` fails due to a channel ban
 - NickServ `RETURN` now enables the `HIDEMAIL` flag if the email was changed
   (unless the flag is unset by default)
@@ -221,9 +247,9 @@ Build System
 Password Cryptography
 ---------------------
 - The existing crypto modules no longer need OpenSSL (or any crypto library)
-- Add support for scrypt password encryption with `modules/crypto/scrypt`.
+- Add support for scrypt password encryption with `crypto/scrypt`.
   The scrypt module requires libsodium (`--with-sodium`).
-- Add support for bcrypt password encryption with `modules/crypto/bcrypt`.
+- Add support for bcrypt password encryption with `crypto/bcrypt`.
 - `libathemecore/crypto.c`: log current crypto provider on mod(un/re)load
 - `libathemecore/crypto.c`: rip out plaintext fallback implementation
 - Make old modules (`ircservices`, `pbkdf2`, `rawmd5`, `rawsha1`) verify-only

include/atheme/linker.h

@@ -12,6 +12,6 @@
 
 #include <atheme/stdheaders.h>
 
-mowgli_module_t *linker_open_ext(const char *path, char *errbuf, int errlen);
+mowgli_module_t *linker_open_ext(const char *path, char *errbuf, size_t errlen);
 
 #endif /* !ATHEME_INC_LINKER_H */

include/atheme/module.h

@@ -45,17 +45,18 @@ struct module
 	enum module_unload_capability   can_unload;
 	unsigned int                    mflags;
 	const struct v4_moduleheader *  header;
-	void *                          address;
+	const void *                    address;
 	mowgli_module_t *               handle;
 	void                          (*unload_handler)(struct module *, enum module_unload_intent);
-	mowgli_list_t                   dephost;
-	mowgli_list_t                   deplist;
-	mowgli_list_t                   symlist;
+	mowgli_list_t                   required_by;
+	mowgli_list_t                   requires;
+	mowgli_node_t                   mbl_node;
+	mowgli_node_t                   mod_node;
 };
 
 struct v4_moduleheader
 {
-	unsigned int                    atheme_mod;
+	unsigned int                    magic;
 	unsigned int                    abi_ver;
 	unsigned int                    abi_rev;
 	const char *                    serial;
@@ -67,16 +68,8 @@ struct v4_moduleheader
 	const char *                    version;
 };
 
-struct module_dependency
-{
-	char *                          name;
-	enum module_unload_capability   can_unload;
-};
-
 void modules_init(void);
 struct module *module_load(const char *filespec);
-void module_load_dir(const char *dirspec);
-void module_load_dir_match(const char *dirspec, const char *pattern);
 void *module_locate_symbol(const char *modname, const char *sym);
 void module_unload(struct module *m, enum module_unload_intent intent);
 struct module *module_find(const char *name);
@@ -86,19 +79,19 @@ bool module_request(const char *name);
 // Located in libathemecore/module.c
 extern mowgli_list_t modules;
 
-#define DECLARE_MODULE_V1(_name, _unloadcap, _modinit, _moddeinit, _ver, _ven)  \
-        extern const struct v4_moduleheader _header;                            \
-        const struct v4_moduleheader _header = {                                \
-                .atheme_mod = MAPI_ATHEME_MAGIC,                                \
-                .abi_ver    = MAPI_ATHEME_V4,                                   \
-                .abi_rev    = CURRENT_ABI_REVISION,                             \
-                .serial     = SERNO,                                            \
-                .name       = _name,                                            \
-                .can_unload = _unloadcap,                                       \
-                .modinit    = _modinit,                                         \
-                .deinit     = _moddeinit,                                       \
-                .vendor     = _ven,                                             \
-                .version    = _ver,                                             \
+#define DECLARE_MODULE_V1(_name, _ucap, _modinit, _moddeinit, _ver, _ven)  \
+        extern const struct v4_moduleheader _header;                       \
+        const struct v4_moduleheader _header = {                           \
+                .magic      = MAPI_ATHEME_MAGIC,                           \
+                .abi_ver    = MAPI_ATHEME_V4,                              \
+                .abi_rev    = CURRENT_ABI_REVISION,                        \
+                .serial     = SERNO,                                       \
+                .name       = _name,                                       \
+                .can_unload = _ucap,                                       \
+                .modinit    = _modinit,                                    \
+                .deinit     = _moddeinit,                                  \
+                .vendor     = _ven,                                        \
+                .version    = _ver,                                        \
         }
 
 #define VENDOR_DECLARE_MODULE_V1(name, unloadcap, ven)                     \

include/atheme/structures.h

@@ -71,7 +71,6 @@ struct atheme_regex;
 
 // Defined in atheme/module.h
 struct module;
-struct module_dependency;
 struct v4_moduleheader;
 
 // Defined in atheme/object.h

libathemecore/conf.c

@@ -319,31 +319,62 @@ init_newconf(void)
 static int
 c_loadmodule(mowgli_config_file_entry_t *ce)
 {
-	char pathbuf[4096];
-	char *name;
-
 	if (!cold_start)
 		return 0;
 
-	if (ce->vardata == NULL)
+	// loadmodule "foo/bar";
+	if (strcmp(ce->varname, "loadmodule") == 0 && ! ce->entries && ! ce->prevlevel)
 	{
-		conf_report_warning(ce, "no parameter for configuration option");
+		if (ce->vardata)
+			(void) module_load(ce->vardata);
+		else
+			(void) conf_report_warning(ce, "no parameter for configuration option");
+
 		return 0;
 	}
 
-	name = ce->vardata;
-
-	if (*name == '/')
+	/* loadmodule {
+	 *   foo {
+	 *     bar;
+	 *   };
+	 * };
+	 */
+	if (ce->entries)
 	{
-		module_load(name);
+		MOWGLI_ITER_FOREACH(ce, ce->entries)
+			(void) c_loadmodule(ce);
+
 		return 0;
 	}
-	else
+
+	char path[PATH_MAX] = { 0 };
+	char ptmp[PATH_MAX] = { 0 };
+
+	while (ce->prevlevel)
 	{
-		snprintf(pathbuf, 4096, "%s/%s", MODDIR, name);
-		module_load(pathbuf);
-		return 0;
+		if (ce->vardata || ! ce->varname)
+		{
+			(void) conf_report_warning(ce, "invalid parameter for configuration option");
+			return 0;
+		}
+
+		(void) mowgli_strlcpy(ptmp, ce->varname, sizeof ptmp);
+
+		if (*path)
+		{
+			(void) mowgli_strlcat(ptmp, "/", sizeof ptmp);
+			(void) mowgli_strlcat(ptmp, path, sizeof ptmp);
+		}
+
+		(void) mowgli_strlcpy(path, ptmp, sizeof path);
+
+		ce = ce->prevlevel;
 	}
+
+	if (*path)
+		(void) module_load(path);
+
+	return 0;
 }
 
 static int

libathemecore/linker.c

@@ -37,7 +37,7 @@
  *       a shared module is loaded into the application's memory space
  */
 mowgli_module_t *
-linker_open_ext(const char *path, char *errbuf, int errlen)
+linker_open_ext(const char *path, char *errbuf, size_t errlen)
 {
 	size_t len = strlen(path) + 20;
 	char *buf = smalloc(len);

modules/operserv/Makefile

@@ -27,11 +27,8 @@ SRCS   =                    \
     jupe.c                  \
     main.c                  \
     mode.c                  \
-    modinspect.c            \
     modlist.c               \
-    modload.c               \
-    modreload.c             \
-    modunload.c             \
+    modmanager.c            \
     noop.c                  \
     rakill.c                \
     raw.c                   \

modules/operserv/modinspect.c

-/*
- * SPDX-License-Identifier: ISC
- * SPDX-URL: https://spdx.org/licenses/ISC.html
- *
- * Copyright (C) 2005-2006 Atheme Project (http://atheme.org/)
- *
- * A simple module inspector.
- */
-
-#include <atheme.h>
-
-static void
-os_cmd_modinspect(struct sourceinfo *si, int parc, char *parv[])
-{
-	char *mname = parv[0];
-	struct module *m;
-
-	if (!mname)
-	{
-		command_fail(si, fault_needmoreparams, STR_INSUFFICIENT_PARAMS, "MODINSPECT");
-		command_fail(si, fault_needmoreparams, _("Syntax: MODINSPECT <module>"));
-		return;
-	}
-
-	logcommand(si, CMDLOG_GET, "MODINSPECT: \2%s\2", mname);
-
-	m = module_find_published(mname);
-
-	if (!m)
-	{
-		command_fail(si, fault_nosuch_target, _("\2%s\2 is not loaded."), mname);
-		return;
-	}
-
-	// Is there a header?
-	if (!m->header)
-	{
-		command_fail(si, fault_unimplemented, _("\2%s\2 cannot be inspected."), mname);
-		return;
-	}
-
-	command_success_nodata(si, _("Information on \2%s\2:"), mname);
-	command_success_nodata(si, _("Name       : %s"), m->name);
-	command_success_nodata(si, _("Address    : %p"), m->address);
-	command_success_nodata(si, _("Entry point: %p"), m->header->modinit);
-	command_success_nodata(si, _("Exit point : %p"), m->header->deinit);
-	command_success_nodata(si, _("SDK Serial : %s"), m->header->serial);
-	command_success_nodata(si, _("Version    : %s"), m->header->version);
-	command_success_nodata(si, _("Vendor     : %s"), m->header->vendor);
-	command_success_nodata(si, _("Can unload : %s"), m->can_unload == MODULE_UNLOAD_CAPABILITY_OK ? _("Yes") :
-	                           (m->can_unload == MODULE_UNLOAD_CAPABILITY_NEVER ? _("No") : _("Reload-only")));
-	command_success_nodata(si, _("*** \2End of Info\2 ***"));
-}
-
-static struct command os_modinspect = {
-	.name           = "MODINSPECT",
-	.desc           = N_("Displays information about loaded modules."),
-	.access         = PRIV_SERVER_AUSPEX,
-	.maxparc        = 1,
-	.cmd            = &os_cmd_modinspect,
-	.help           = { .path = "oservice/modinspect" },
-};
-
-static void
-mod_init(struct module *const restrict m)
-{
-	MODULE_TRY_REQUEST_DEPENDENCY(m, "operserv/main")
-
-	service_named_bind_command("operserv", &os_modinspect);
-}
-
-static void
-mod_deinit(const enum module_unload_intent ATHEME_VATTR_UNUSED intent)
-{
-	service_named_unbind_command("operserv", &os_modinspect);
-}
-
-SIMPLE_DECLARE_MODULE_V1("operserv/modinspect", MODULE_UNLOAD_CAPABILITY_OK)

modules/operserv/modload.c

-/*
- * SPDX-License-Identifier: ISC
- * SPDX-URL: https://spdx.org/licenses/ISC.html
- *
- * Copyright (C) 2005-2006 William Pitcock, et al.
- *
- * Loads a new module in.
- */
-
-#include <atheme.h>
-
-static void
-os_cmd_modload(struct sourceinfo *si, int parc, char *parv[])
-{
-	char *module;
-	struct module *m;
-	int i;
-
-	if (parc < 1)
-	{
-		command_fail(si, fault_needmoreparams, STR_INSUFFICIENT_PARAMS, "MODLOAD");
-		command_fail(si, fault_needmoreparams, _("Syntax: MODLOAD <module...>"));
-		return;
-	}
-	i = 0;
-	while (i < parc)
-	{
-		module = parv[i++];
-		if (module_find_published(module))
-		{
-			command_fail(si, fault_nochange, _("\2%s\2 is already loaded."), module);
-			continue;
-		}
-
-		logcommand(si, CMDLOG_ADMIN, "MODLOAD: \2%s\2", module);
-		m = module_load(module);
-
-		if (m != NULL)
-			command_success_nodata(si, _("Module \2%s\2 loaded."), module);
-		else
-			command_fail(si, fault_nosuch_target, _("Module \2%s\2 failed to load."), module);
-	}
-
-	if (conf_need_rehash)
-	{
-		logcommand(si, CMDLOG_ADMIN, "REHASH (MODLOAD)");
-		wallops("Rehashing \2%s\2 to complete module load by request of \2%s\2.", config_file, get_oper_name(si));
-		if (!conf_rehash())
-			command_fail(si, fault_nosuch_target, _("REHASH of \2%s\2 failed. Please correct any errors in the file and try again."), config_file);
-	}
-}
-
-static struct command os_modload = {
-	.name           = "MODLOAD",
-	.desc           = N_("Loads a module."),
-	.access         = PRIV_ADMIN,
-	.maxparc        = 20,
-	.cmd            = &os_cmd_modload,
-	.help           = { .path = "oservice/modload" },
-};
-
-static void
-mod_init(struct module *const restrict m)
-{
-	MODULE_TRY_REQUEST_DEPENDENCY(m, "operserv/main")
-
-	service_named_bind_command("operserv", &os_modload);
-}
-
-static void
-mod_deinit(const enum module_unload_intent ATHEME_VATTR_UNUSED intent)
-{
-	service_named_unbind_command("operserv", &os_modload);
-}
-
-SIMPLE_DECLARE_MODULE_V1("operserv/modload", MODULE_UNLOAD_CAPABILITY_OK)

modules/operserv/modreload.c

-/*
- * SPDX-License-Identifier: ISC
- * SPDX-URL: https://spdx.org/licenses/ISC.html
- *
- * Copyright (C) 2009 Michael Rodriguez <dkingston@dkingston.net>
- *
- * Reloads a module.
- */
-
-#include <atheme.h>
-
-static void
-recurse_module_deplist(struct module *m, mowgli_list_t *deplist)
-{
-	mowgli_node_t *n;
-	MOWGLI_LIST_FOREACH(n, m->dephost.head)
-	{
-		struct module *dm = (struct module *) n->data;
-
-		// Skip duplicates
-		bool found = false;
-		mowgli_node_t *n2;
-		MOWGLI_LIST_FOREACH(n2, deplist->head)
-		{
-			struct module_dependency *existing_dep = n2->data;
-			if (0 == strcasecmp(dm->name, existing_dep->name))
-				found = true;
-		}
-		if (found)
-			continue;
-
-		struct module_dependency *const dep = smalloc(sizeof *dep);
-		dep->name = sstrdup(dm->name);
-		dep->can_unload = dm->can_unload;
-		mowgli_node_add(dep, mowgli_node_create(), deplist);
-
-		recurse_module_deplist(dm, deplist);
-	}
-}
-
-static void
-os_cmd_modreload(struct sourceinfo *si, int parc, char *parv[])
-{
-	char *module = parv[0];
-	struct module *m;
-	mowgli_node_t *n;
-	struct module_dependency * reloading_semipermanent_module = NULL;
-
-	if (parc < 1)
-        {
-		command_fail(si, fault_needmoreparams, STR_INSUFFICIENT_PARAMS, "MODRELOAD");
-		command_fail(si, fault_needmoreparams, _("Syntax: MODRELOAD <module...>"));
-		return;
-        }
-
-	if (!(m = module_find_published(module)))
-	{
-		command_fail(si, fault_nochange, _("\2%s\2 is not loaded."), module);
-		return;
-	}
-
-	/* Make sure these checks happen before we construct the list of dependent modules, so that
-	 * we can return without having to clean up everything.
-	 */
-
-	if (!strcmp(m->name, "operserv/main") || !strcmp(m->name, "operserv/modload") || !strcmp(m->name, "operserv/modunload") || !strcmp(m->name, "operserv/modreload"))
-	{
-		command_fail(si, fault_noprivs, _("Refusing to reload \2%s\2."), module);
-		return;
-	}
-
-	if (m->can_unload == MODULE_UNLOAD_CAPABILITY_NEVER)
-	{
-		command_fail(si, fault_noprivs, _("\2%s\2 is a permanent module; it cannot be reloaded."), module);
-		slog(LG_ERROR, "MODRELOAD:ERROR: \2%s\2 tried to reload permanent module \2%s\2", get_oper_name(si), module);
-		return;
-	}
-
-	mowgli_list_t *module_deplist = mowgli_list_create();
-	struct module_dependency *const self_dep = smalloc(sizeof *self_dep);
-	self_dep->name = sstrdup(module);
-	self_dep->can_unload = m->can_unload;
-	mowgli_node_add(self_dep, mowgli_node_create(), module_deplist);
-	recurse_module_deplist(m, module_deplist);
-
-	MOWGLI_LIST_FOREACH(n, module_deplist->head)
-	{
-		struct module_dependency *dep = n->data;
-		if (dep->can_unload == MODULE_UNLOAD_CAPABILITY_NEVER)
-		{
-			command_fail(si, fault_noprivs, _("\2%s\2 is depended upon by \2%s\2, which is a permanent module and cannot be reloaded."), module, dep->name);
-			slog(LG_ERROR, "MODRELOAD:ERROR: \2%s\2 tried to reload \2%s\2, which is depended upon by permanent module \2%s\2", get_oper_name(si), module, dep->name);
-
-			// We've constructed the dep list now, so don't return without cleaning up
-			while (module_deplist->head != NULL)
-			{
-				dep = module_deplist->head->data;
-				sfree(dep->name);
-				sfree(dep);
-				mowgli_node_delete(module_deplist->head, module_deplist);
-				mowgli_list_free(module_deplist);
-				return;
-			}
-		}
-		else if (dep->can_unload == MODULE_UNLOAD_CAPABILITY_RELOAD_ONLY
-			&& reloading_semipermanent_module == NULL)
-		{
-			reloading_semipermanent_module = dep;
-		}
-	}
-
-	if (reloading_semipermanent_module)
-	{
-		/* If we're reloading a semi-permanent module (reload only; no unload), then there's
-		 * a chance that we'll abort if the module fails to load again. Save the DB beforehand
-		 * just in case
-		 */
-		slog(LG_INFO, "UPDATE (due to reload of module \2%s\2): \2%s\2",
-				reloading_semipermanent_module->name, get_oper_name(si));
-		wallops("Updating database by request of \2%s\2.", get_oper_name(si));
-		db_save(NULL, DB_SAVE_BG_IMPORTANT);
-	}
-
-	module_unload(m, MODULE_UNLOAD_INTENT_RELOAD);
-
-	while (module_deplist->head != NULL)
-	{
-		struct module *t;
-		n = module_deplist->head;
-		struct module_dependency *dep = n->data;
-
-		t = module_load(dep->name);
-
-		if (t != NULL)
-		{
-			logcommand(si, CMDLOG_ADMIN, "MODRELOAD: \2%s\2 (from \2%s\2)", dep->name, module);
-			command_success_nodata(si, _("Module \2%s\2 reloaded (from \2%s\2)."), dep->name, module);
-		}
-		else
-		{
-			if (dep->can_unload != MODULE_UNLOAD_CAPABILITY_OK)
-			{
-				// Failed to reload a module that can't be unloaded. Abort.
-				command_fail(si, fault_nosuch_target, _(
-						"Module \2%s\2 failed to reload, and does not allow unloading. "
-						"Shutting down to avoid data loss."), dep->name);
-				slog(LG_ERROR, "MODRELOAD:ERROR: \2%s\2 failed to reload and does not allow unloading. "
-						"Shutting down to avoid data loss.", dep->name);
-				sendq_flush(curr_uplink->conn);
-
-				exit(EXIT_FAILURE);