Skip to contents

Reformat a .rda file which has a matrix with terms as row names, or a plain-text embeddings file which has a term at the start of each line, and consistent delimiting characters. Plain-text files are processed line-by-line, so large spaces can be reformatted RAM-conservatively.


standardize.lspace(infile, name, sep = " ", digits = 9,
  dir = getOption("lingmatch.lspace.dir"), outdir = dir, remove = "",
  term_check = "^[a-zA-Z]+$|^['a-zA-Z][a-zA-Z.'\\/-]*[a-zA-Z.]$",
  verbose = FALSE)



Name of the .rda or plain-text file relative to dir,
e.g., "default.rda" or "glove/glove.6B.300d.txt".


Base name of the reformatted file and term file; e.g., "glove" would result in glove.dat and glove_terms.txt in outdir.


Delimiting character between values in each line, e.g., " " or "\t". Only applies to plain-text files.


Number of digits to round values to; default is 9.


Path to folder containing infiles.
Default is getOption('lingmatch.lspace.dir'), which must be set in the current session. If this is not specified and infile is a full path, dir will be set to infile's parent directory.


Path to folder in which to save standardized files; default is dir.


A string with a regex pattern to be removed from term names
(i.e., gsub(remove, "", term)); default is "", which is ignored.


A string with a regex pattern by which to filter terms; i.e., only lines with fully matched terms are written to the reformatted file. The default attempts to retain only regular words, including those with dashes, foreword slashes, and periods. Set to an empty string ("") to write all lines regardless of term.


Logical: if TRUE, prints the current line number and its term to the console every 1,000 lines. Only applies to plain-text files.


Path to the standardized [1] data file and [2] terms file if applicable.

See also

Other Latent Semantic Space functions: download.lspace(), lma_lspace(), select.lspace()


if (FALSE) {

# from
standardize.lspace("EN_100k_lsa.rda", "100k_lsa")

# from
standardize.lspace("crawl-300d-2M.vec", "facebook_crawl")

# Standardized versions of these spaces can also be downloaded with download.lspace.